-- Logs begin at Mon 2022-08-29 21:48:27 UTC, end at Mon 2022-08-29 23:07:56 UTC. -- Aug 29 22:27:28.523156 dv-cinder-854269-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 29 22:27:28.535543 dv-cinder-854269-4-iscsi devstack@keystone.service[93259]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 29 22:27:28.535543 dv-cinder-854269-4-iscsi devstack@keystone.service[93259]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Aug 29 22:27:28 2022] *** Aug 29 22:27:28.535543 dv-cinder-854269-4-iscsi devstack@keystone.service[93259]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 29 22:27:28.535543 dv-cinder-854269-4-iscsi devstack@keystone.service[93259]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 29 22:27:28.535543 dv-cinder-854269-4-iscsi devstack@keystone.service[93259]: nodename: dv-cinder-854269-4-iscsi Aug 29 22:27:28.535543 dv-cinder-854269-4-iscsi devstack@keystone.service[93259]: machine: x86_64 Aug 29 22:27:28.535543 dv-cinder-854269-4-iscsi devstack@keystone.service[93259]: clock source: unix Aug 29 22:27:28.535543 dv-cinder-854269-4-iscsi devstack@keystone.service[93259]: pcre jit disabled Aug 29 22:27:28.535543 dv-cinder-854269-4-iscsi devstack@keystone.service[93259]: detected number of CPU cores: 4 Aug 29 22:27:28.535543 dv-cinder-854269-4-iscsi devstack@keystone.service[93259]: current working directory: / Aug 29 22:27:28.535543 dv-cinder-854269-4-iscsi devstack@keystone.service[93259]: detected binary path: /usr/bin/uwsgi-core Aug 29 22:27:28.535543 dv-cinder-854269-4-iscsi devstack@keystone.service[93259]: your processes number limit is 47680 Aug 29 22:27:28.535543 dv-cinder-854269-4-iscsi devstack@keystone.service[93259]: your memory page size is 4096 bytes Aug 29 22:27:28.535543 dv-cinder-854269-4-iscsi devstack@keystone.service[93259]: detected max file descriptor number: 2048 Aug 29 22:27:28.535543 dv-cinder-854269-4-iscsi devstack@keystone.service[93259]: lock engine: pthread robust mutexes Aug 29 22:27:28.535543 dv-cinder-854269-4-iscsi devstack@keystone.service[93259]: thunder lock: enabled Aug 29 22:27:28.536989 dv-cinder-854269-4-iscsi devstack@keystone.service[93259]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 29 22:27:28.536989 dv-cinder-854269-4-iscsi devstack@keystone.service[93259]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 29 22:27:28.574897 dv-cinder-854269-4-iscsi devstack@keystone.service[93259]: Python main interpreter initialized at 0x55d68c5b59c0 Aug 29 22:27:28.574897 dv-cinder-854269-4-iscsi devstack@keystone.service[93259]: python threads support enabled Aug 29 22:27:28.574897 dv-cinder-854269-4-iscsi devstack@keystone.service[93259]: your server socket listen backlog is limited to 100 connections Aug 29 22:27:28.574897 dv-cinder-854269-4-iscsi devstack@keystone.service[93259]: your mercy for graceful operations on workers is 90 seconds Aug 29 22:27:28.575547 dv-cinder-854269-4-iscsi devstack@keystone.service[93259]: mapped 403077 bytes (393 KB) for 2 cores Aug 29 22:27:28.575635 dv-cinder-854269-4-iscsi devstack@keystone.service[93259]: *** Operational MODE: preforking *** Aug 29 22:27:28.575921 dv-cinder-854269-4-iscsi devstack@keystone.service[93259]: *** uWSGI is running in multiple interpreter mode *** Aug 29 22:27:28.575921 dv-cinder-854269-4-iscsi devstack@keystone.service[93259]: spawned uWSGI master process (pid: 93259) Aug 29 22:27:28.576125 dv-cinder-854269-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 29 22:27:28.577187 dv-cinder-854269-4-iscsi devstack@keystone.service[93259]: spawned uWSGI worker 1 (pid: 93271, cores: 1) Aug 29 22:27:28.577187 dv-cinder-854269-4-iscsi devstack@keystone.service[93259]: spawned uWSGI worker 2 (pid: 93272, cores: 1) Aug 29 22:27:28.580010 dv-cinder-854269-4-iscsi devstack@keystone.service[93259]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 29 22:27:32.957005 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 29 22:27:32.957005 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: from cryptography.utils import int_from_bytes Aug 29 22:27:32.957862 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 29 22:27:32.957862 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: from cryptography.utils import int_from_bytes Aug 29 22:27:32.958315 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 29 22:27:32.958315 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: from cryptography.utils import int_from_bytes Aug 29 22:27:32.958423 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 29 22:27:32.958423 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: from cryptography.utils import int_from_bytes Aug 29 22:27:33.008364 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 29 22:27:33.008572 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 29 22:27:33.008747 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] command line args: [] {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 29 22:27:33.009021 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 29 22:27:33.009204 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ================================================================================ {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 29 22:27:33.009459 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] admin_token = **** {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.009703 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] config_dir = [] {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.009891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.010071 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] config_source = [] {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.010252 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] debug = True {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.010454 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.011661 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.011661 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] insecure_debug = False {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.011795 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 29 22:27:33.011864 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.011929 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 29 22:27:33.011992 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.012061 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] command line args: [] {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 29 22:27:33.012119 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] list_limit = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.012180 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 29 22:27:33.012243 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] log_config_append = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.012300 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ================================================================================ {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 29 22:27:33.012355 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.012355 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] log_dir = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.012467 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] admin_token = **** {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.012528 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] log_file = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.012595 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] config_dir = [] {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.012658 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.012753 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.012830 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] config_source = [] {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.012894 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.012957 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] debug = True {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.013069 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.013184 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.013299 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.013382 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.013483 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.013615 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] insecure_debug = False {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.013742 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.013819 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.013945 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.014021 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.014141 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.014246 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] list_limit = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.014345 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.014433 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] log_config_append = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.014579 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.014654 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.014763 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] log_dir = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.014838 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.014958 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] log_file = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.015044 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.015685 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.015923 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.016059 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.016164 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] notification_format = cadf {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.016361 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.016496 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.016801 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.021506 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.021506 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] publish_errors = False {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.021736 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.021811 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.021927 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.022007 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.022007 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.022145 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.022224 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.022224 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.022362 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.022438 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] standard_threads = False {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.022513 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.022646 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] strict_password_check = False {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.022725 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.022802 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.022802 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] use_eventlog = False {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.022931 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.023001 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] use_journal = False {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.023001 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] use_json = False {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.023136 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.023205 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] use_stderr = False {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.023205 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] use_syslog = False {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.023396 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.023525 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] watch_log_file = False {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.023525 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.023721 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] notification_format = cadf {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.023839 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.023839 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.023968 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.024046 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.024046 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.024046 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.024231 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.024302 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.024302 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.024532 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] publish_errors = False {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.024606 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.024682 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.024803 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.024803 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.025015 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.025086 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.025161 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.025236 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.025236 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.025367 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.025438 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.025513 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.025588 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.025658 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] standard_threads = False {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.025732 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.025812 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] strict_password_check = False {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.025884 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.025884 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.026015 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.026089 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.026159 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] use_eventlog = False {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.026235 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.026235 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.026360 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] use_journal = False {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.026434 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.026504 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] use_json = False {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.026580 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.026651 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] use_stderr = False {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.026651 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] use_syslog = False {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.026774 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.026843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] watch_log_file = False {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:27:33.026912 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.026987 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.027047 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.027047 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.027216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.027291 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.027366 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.027447 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.027524 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.027595 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.027664 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.027732 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.027800 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.027877 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.027948 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.028022 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.028093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.028162 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.028230 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.028376 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.028450 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.028524 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.028600 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.028670 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.028760 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.028847 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.028990 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.029075 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.029235 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.029317 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.029488 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.029573 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.029722 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.029804 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.029955 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.030031 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.030193 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.030280 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.030443 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.030526 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.030673 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.030838 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.031441 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.031564 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.031663 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.031903 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.032013 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.032139 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.032383 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.032497 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.032602 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.032737 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.032861 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.032940 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.033100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.033191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.033341 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.033482 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.033557 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.033673 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.033782 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.033867 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.034015 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.034094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.034265 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.034343 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.034501 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.034582 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.034740 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.034821 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.034973 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.035058 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.035229 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.035313 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.035463 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.035541 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.035695 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.035775 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.035934 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.036053 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.036171 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.036250 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.036411 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.036497 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.036648 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.036745 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.036913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.036992 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.037152 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.037270 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.037386 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.037470 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.037623 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.037701 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.037878 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.037960 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.038114 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.038189 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] cache.enabled = True {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.038359 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.038440 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.038594 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.038673 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.038843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.038954 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.039075 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.039158 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.039327 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.039412 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.039560 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.039637 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.039799 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.039878 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.040036 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.040114 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.040272 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.040352 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.040511 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.040619 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.040770 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.040853 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.041012 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.041173 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.041248 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.041318 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.041494 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.041578 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.041734 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.041822 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.041972 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.042049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.042205 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.042282 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.042443 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.042529 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.042681 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.042766 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.042913 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.042991 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.043146 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.043230 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.043380 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.043511 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.043619 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.043697 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.043850 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.043926 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.044101 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.044188 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.044330 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.044406 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.044574 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.044686 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.044835 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.044925 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.045090 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] cache.enabled = True {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.045166 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.045591 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.045682 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.045834 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.045919 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.046069 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] auth.external = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.046201 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.046312 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] auth.mapped = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.046394 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.046619 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.046705 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.046862 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.047004 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.047116 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] auth.password = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.047245 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.047322 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] auth.token = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.047489 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.047568 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.047731 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.047808 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] catalog.caching = True {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.047980 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.048066 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.048207 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.048284 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.048454 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.048538 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.048704 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.048804 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.048962 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.049040 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.049253 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.049445 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] credential.caching = True {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.049520 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.049622 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] credential.driver = sql {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.049703 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.049778 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.049924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.050005 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.050166 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.050250 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.050406 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.050483 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.050643 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.050722 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.050880 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.050964 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.051123 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.051201 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.051359 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.051438 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.051603 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.051687 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.051844 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.051921 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.052100 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.052178 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.052343 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.052428 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.052601 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.052678 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.052884 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.052965 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.053123 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] federation.caching = True {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.053202 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.053359 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] auth.external = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.053437 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.053600 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] auth.mapped = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.053686 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] federation.driver = sql {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.053843 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.053924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.054076 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.054162 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.054325 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.054403 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] auth.password = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.054563 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.054648 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] auth.token = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.054803 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.054956 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.055032 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.055102 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] catalog.caching = True {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.055282 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.055440 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.055504 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.055572 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.055750 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.055837 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.055985 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] identity.caching = True {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.056065 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.056217 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.056332 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.056443 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] credential.caching = True {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.056539 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.056693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.056848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] credential.driver = sql {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.056954 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.057042 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.057205 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] identity.driver = sql {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.057283 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.057437 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.057514 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.057677 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.057765 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.057939 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.058061 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.058169 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.058253 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.058404 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.058493 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.058638 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.058715 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.058871 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.058950 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.059123 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.059209 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.059372 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.059450 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.059620 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.059698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.059876 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.059962 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.060125 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.060203 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.060362 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] federation.caching = True {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.060438 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.060603 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.060693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.060873 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.061001 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] federation.driver = sql {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.061127 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.061252 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.061386 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.061531 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.061659 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.061784 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.061910 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.062048 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.062181 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.062333 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.062455 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.062590 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.062777 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.062970 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.063098 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.063234 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.063373 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.063499 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.063644 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.063771 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] identity.caching = True {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.063893 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.064026 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.064152 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.064276 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.064407 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.064532 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.064665 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.064807 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.064941 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.065062 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] identity.driver = sql {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.065194 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.password = **** {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.065315 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.065446 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.065571 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.065699 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.065820 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.065942 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.065942 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.066176 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.066297 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.066417 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.066537 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.066657 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.066657 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.066903 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.066903 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.067139 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.067261 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.067381 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.067501 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.067629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.067769 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.067896 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.067973 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.068045 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.068116 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.068196 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.068270 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.068341 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.068412 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.user = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.068482 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.068556 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.068683 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.068783 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.068868 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.068942 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.069013 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.069084 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.069163 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.069236 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.069314 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.069452 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.069552 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.069689 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.069808 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.069944 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.070050 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.070129 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.070366 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.070460 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.070543 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.070617 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.070780 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.070859 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.071030 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.071114 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.071275 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.071353 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.071507 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.071584 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.071748 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.071831 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.071983 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.password = **** {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.072063 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.072212 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.072288 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.072453 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.072531 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.072688 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.072831 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.072949 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.073027 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.073186 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.073264 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.075024 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.075196 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.075284 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.075356 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.075428 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.075499 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.075580 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.075654 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.075725 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] policy.driver = sql {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.075796 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.075867 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.075938 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.076018 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.076090 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.076162 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.076232 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.076304 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] receipt.caching = True {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.076374 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.076453 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.076526 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.user = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.076526 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.076653 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.076745 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.076822 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.076905 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.076979 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.077050 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.077121 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.077191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.077270 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] resource.caching = True {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.077348 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.077421 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.077492 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.077562 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] resource.driver = sql {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.077632 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.078435 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.078566 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.078664 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.078739 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.078811 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.078882 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.078963 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] revoke.caching = True {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.079036 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.079107 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.079178 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.079319 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.079398 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.079566 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.079644 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] role.cache_time = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.079805 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] role.caching = True {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.079918 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.080052 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.080130 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] role.driver = sql {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.080285 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] role.list_limit = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.080403 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.080534 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.080611 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.080797 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.080878 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.081031 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.081114 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.081270 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.081347 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.081509 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.081586 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.081755 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.081834 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.082004 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.082080 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.082244 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.082321 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.082477 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.082557 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] policy.driver = sql {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.082728 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.082805 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.082972 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.083048 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.083213 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.083292 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.083451 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] receipt.caching = True {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.083530 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.083693 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.083874 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.083960 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.084114 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.084191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.084365 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.084469 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.084617 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.084704 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.084886 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] resource.caching = True {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.084964 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.085135 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.085247 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.085371 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.085462 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] resource.driver = sql {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.085612 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.085697 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.085865 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.085942 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.086106 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.086192 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.086340 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.086418 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] revoke.caching = True {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.086574 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.086651 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.086807 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.086892 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.087047 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] role.cache_time = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.087389 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.087472 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] role.caching = True {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.087545 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.087625 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] role.driver = sql {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.087698 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.087770 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] role.list_limit = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.087841 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.087978 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.088056 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.088212 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] token.cache_time = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.088291 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.088450 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] token.caching = True {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.088534 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.088690 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.088799 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.088967 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.089046 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] token.provider = fernet {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.089210 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.089287 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.089458 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.089535 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.089702 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.089780 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.089938 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.090014 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.090179 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.090256 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.090423 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.090502 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.090670 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] trust.driver = sql {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.090748 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.090907 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.090985 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.091146 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.091319 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.091483 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.091661 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.091740 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.091813 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.091928 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.092037 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.092160 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.092292 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.092404 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.092548 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.092674 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.092818 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.092962 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] database.connection = **** {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.093080 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.093207 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.093408 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.094157 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.094341 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.094483 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.094578 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.094653 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.094820 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.094899 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.095058 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.095139 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.095297 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.095375 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.095536 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.095613 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.095775 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.095855 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.096016 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.096093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.096252 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.096329 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] token.cache_time = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.096487 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] token.caching = True {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.096565 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.096763 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.096844 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.096996 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.097073 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] token.provider = fernet {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.097235 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.097313 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.097478 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.097555 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.097712 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.097846 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.097952 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.098032 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.098189 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.098268 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.098361 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 29 22:27:33.098439 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.098670 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] trust.driver = sql {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.098912 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.099156 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.099395 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.099648 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.099928 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.100165 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.100414 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.100685 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.100976 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] database.connection = **** {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.101226 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.101479 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.103658 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.103658 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.103658 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.103658 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.103658 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.103658 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.103658 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.103658 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.103658 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.104183 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.104183 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.104301 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.104463 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.104700 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.104969 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.105211 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:27:33.105386 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 29 22:27:33.127667 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=93272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.127899 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=93272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.128127 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=93272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.130459 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=93271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.130698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=93271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.130932 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=93271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.136545 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=93272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.139380 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=93271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.165685 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=93272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.165914 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=93272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.166755 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=93271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.166994 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=93271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.226098 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=93271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.227979 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=93272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.276065 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=93271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.276290 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=93271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.280240 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=93272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.280480 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=93272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.435753 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.435994 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=93272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.436206 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=93272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.436426 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=93272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.436645 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=93272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.436887 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=93272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.437380 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.437792 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=93271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.438034 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=93271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.438262 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=93271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.438487 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=93271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.438710 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=93271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.459893 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.460119 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=93272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.460346 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=93272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.460495 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.460617 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=93272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.460754 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=93271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.460847 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=93272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.460989 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=93271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.461074 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=93272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.461214 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=93271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.461297 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.461432 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=93271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.461649 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=93271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.461868 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.472878 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=93272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.473535 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=93271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.490650 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.490820 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.490909 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=93271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.491044 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=93271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.491394 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=93272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.491480 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=93271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.491552 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=93272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.491625 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=93271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.491750 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=93272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.491828 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=93271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.491891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=93272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.491952 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.492024 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=93272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.492130 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:27:33.508739 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.508961 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 29 22:27:33.509358 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.509462 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.509605 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 29 22:27:33.513473 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 29 22:27:33.513560 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.513639 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.513639 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 29 22:27:33.513771 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.513842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.513928 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.514003 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.514074 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.514141 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.514141 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.520123 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.520407 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.520684 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.520826 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.520998 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.521109 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.521383 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.521682 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.527905 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=93271) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:27:33.528673 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=93272) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:27:33.531353 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=93271) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:27:33.531644 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.531935 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.532232 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.532404 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=93272) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:27:33.532524 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.532699 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.532801 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.533013 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.533099 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.533300 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.533388 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.533588 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.533723 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.533856 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.534013 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.534127 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.534293 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.534405 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.534682 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.534960 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.535221 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.557568 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.559021 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.559119 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=93271) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:27:33.559322 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.560489 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=93272) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:27:33.560800 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.564190 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=93271) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:27:33.564479 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.564790 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.565402 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=93272) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:27:33.565690 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.565980 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.571112 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=93271) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:27:33.571406 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.572473 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=93272) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:27:33.576357 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.576473 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=93271) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:27:33.576473 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.576473 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.576473 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.576750 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.576842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=93272) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:27:33.576999 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.577081 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.577281 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.577362 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.577563 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.577647 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.577918 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.579959 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.580251 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.580537 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.595562 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.595843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.596615 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.596926 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.599709 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=93271) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:27:33.600777 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=93272) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:27:33.602981 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.603284 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.603854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.604154 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.607509 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=93271) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:27:33.608534 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=93272) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:27:33.611318 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=93271) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:27:33.612358 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=93272) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:27:33.614548 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.614854 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.615145 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.615451 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.615590 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.615738 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.615841 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.616005 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.616090 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.616397 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.616662 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.617072 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.631789 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=93271) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:27:33.632090 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.632362 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.632646 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.632906 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=93272) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:27:33.633201 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.633476 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.633747 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.639531 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.641265 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.641422 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.642180 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.643724 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.644005 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.644478 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.644785 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.646668 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=93272) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:27:33.646954 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.647247 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.647431 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=93271) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:27:33.647528 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.647724 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.647805 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.648131 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.648424 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.648746 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.659594 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=93272) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:27:33.659880 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.660164 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.660448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.660747 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.661015 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=93271) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:27:33.661135 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.661310 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.661428 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.661693 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.661987 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.662287 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.662576 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.662960 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.677546 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=93272) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:27:33.679252 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=93271) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:27:33.680541 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=93272) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:27:33.682201 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=93271) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:27:33.683435 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.684911 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.685129 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.686387 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=93272) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:27:33.686572 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.686699 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.686954 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.688214 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=93271) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:27:33.688510 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.688834 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.693501 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.694975 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=93272) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:27:33.695234 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.696757 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=93271) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:27:33.697903 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.698192 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.698481 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.698768 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.699745 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.700052 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.700346 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.700642 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.707357 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=93272) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:27:33.707650 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.707934 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.709302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=93271) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:27:33.709598 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.709884 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.714858 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=93272) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:27:33.715143 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.715430 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.715713 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.715989 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.716269 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.716545 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.716852 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=93271) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:27:33.716996 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.717089 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.717160 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.717400 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.717501 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.717657 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.717739 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.717921 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.718014 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.718189 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.718309 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.718534 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.718674 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=93272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.718822 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.719177 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.719471 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.719752 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.720021 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.720296 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.720577 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=93271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:27:33.755976 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55d68c5b59c0 pid: 93272 (default app) Aug 29 22:27:33.756633 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55d68c5b59c0 pid: 93271 (default app) Aug 29 22:27:33.759648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d364e3-0f30-4afc-b044-ff1b7814fa0d None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:33.760024 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d364e3-0f30-4afc-b044-ff1b7814fa0d None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:33.760218 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d364e3-0f30-4afc-b044-ff1b7814fa0d None None] PATH_INFO: `/v3` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:33.764585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1/1] 10.222.0.52 () {50 vars in 791 bytes} [Mon Aug 29 22:27:33 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:27:41.542665 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d0346-b6e3-4ac1-927b-3c69487d1580 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:41.543092 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d0346-b6e3-4ac1-927b-3c69487d1580 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:41.543210 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d0346-b6e3-4ac1-927b-3c69487d1580 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:41.549725 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1/2] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:27:41 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:27:41.555342 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c16e26a-9c11-4a28-a63d-79ac8984a471 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:41.555892 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c16e26a-9c11-4a28-a63d-79ac8984a471 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:41.555892 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c16e26a-9c11-4a28-a63d-79ac8984a471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:41.630766 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: INFO dbcounter [None req-6c16e26a-9c11-4a28-a63d-79ac8984a471 None None] Registered counter for database keystone Aug 29 22:27:41.652677 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG oslo_db.sqlalchemy.engines [None req-6c16e26a-9c11-4a28-a63d-79ac8984a471 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=93272) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 29 22:27:41.659324 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writer thread running {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 29 22:27:41.846261 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG passlib.handlers.bcrypt [None req-6c16e26a-9c11-4a28-a63d-79ac8984a471 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=93272) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 29 22:27:41.846571 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG passlib.handlers.bcrypt [None req-6c16e26a-9c11-4a28-a63d-79ac8984a471 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=93272) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 29 22:27:41.908924 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-6c16e26a-9c11-4a28-a63d-79ac8984a471 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:27:41.910526 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-6c16e26a-9c11-4a28-a63d-79ac8984a471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:27:42.047253 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 2/3] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:27:41 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 493 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 29 22:27:42.517069 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866d0346-b6e3-4ac1-927b-3c69487d1580 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:27:42.589166 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: INFO dbcounter [None req-866d0346-b6e3-4ac1-927b-3c69487d1580 None None] Registered counter for database keystone Aug 29 22:27:42.609036 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG oslo_db.sqlalchemy.engines [None req-866d0346-b6e3-4ac1-927b-3c69487d1580 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=93271) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 29 22:27:42.615958 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writer thread running {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 29 22:27:42.736172 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866d0346-b6e3-4ac1-927b-3c69487d1580 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:27:42.740349 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51c9137-ffea-43d4-abe9-8f50d9d4f111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:27:42.740997 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c9137-ffea-43d4-abe9-8f50d9d4f111 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:42.741278 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c9137-ffea-43d4-abe9-8f50d9d4f111 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:42.741551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c9137-ffea-43d4-abe9-8f50d9d4f111 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:42.749677 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51c9137-ffea-43d4-abe9-8f50d9d4f111 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:27:42.802223 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f51c9137-ffea-43d4-abe9-8f50d9d4f111 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 29 22:27:42.802499 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51c9137-ffea-43d4-abe9-8f50d9d4f111 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:27:42.812074 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 2/4] 10.222.0.52 () {58 vars in 1324 bytes} [Mon Aug 29 22:27:42 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 29 22:27:42.818244 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c16e26a-9c11-4a28-a63d-79ac8984a471 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:27:42.833233 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c16e26a-9c11-4a28-a63d-79ac8984a471 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:27:42.837688 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d8fb09-7758-4b88-81c4-7f6675fb3a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:27:42.838782 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8fb09-7758-4b88-81c4-7f6675fb3a01 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:42.839071 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8fb09-7758-4b88-81c4-7f6675fb3a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:42.839349 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8fb09-7758-4b88-81c4-7f6675fb3a01 admin admin] PATH_INFO: `/v3/projects/eb4839200e8e45dfa0acd9e9d8f433bc` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:42.840962 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d8fb09-7758-4b88-81c4-7f6675fb3a01 admin admin] RBAC: Authorizing `identity:get_project(project_id=eb4839200e8e45dfa0acd9e9d8f433bc)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:27:42.898133 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: WARNING keystone.common.rbac_enforcer.enforcer [None req-24d8fb09-7758-4b88-81c4-7f6675fb3a01 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 29 22:27:42.898480 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d8fb09-7758-4b88-81c4-7f6675fb3a01 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:27:42.903405 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 3/5] 10.222.0.52 () {56 vars in 1207 bytes} [Mon Aug 29 22:27:42 2022] GET /identity/v3/projects/eb4839200e8e45dfa0acd9e9d8f433bc => generated 344 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:27:44.821821 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c0920-59ee-4bca-8b3d-db83339ffb7f None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:44.822330 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c0920-59ee-4bca-8b3d-db83339ffb7f None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:44.822330 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c0920-59ee-4bca-8b3d-db83339ffb7f None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:44.826430 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 3/6] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:27:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:27:44.832412 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3866a0-8ff6-4a25-bd29-b20fbc9fd885 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:44.832930 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3866a0-8ff6-4a25-bd29-b20fbc9fd885 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:44.833033 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3866a0-8ff6-4a25-bd29-b20fbc9fd885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:44.852783 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-af3866a0-8ff6-4a25-bd29-b20fbc9fd885 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:27:44.854273 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-af3866a0-8ff6-4a25-bd29-b20fbc9fd885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:27:44.861960 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 4/7] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:27:44 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 29 22:27:45.128455 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c0920-59ee-4bca-8b3d-db83339ffb7f None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:27:45.140541 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c0920-59ee-4bca-8b3d-db83339ffb7f None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:27:45.145192 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df540943-9588-4d6e-9dc3-acdd50ab8b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:27:45.145785 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df540943-9588-4d6e-9dc3-acdd50ab8b7d admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:45.146067 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df540943-9588-4d6e-9dc3-acdd50ab8b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:45.146343 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df540943-9588-4d6e-9dc3-acdd50ab8b7d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:45.153941 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df540943-9588-4d6e-9dc3-acdd50ab8b7d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:27:45.155706 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df540943-9588-4d6e-9dc3-acdd50ab8b7d admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:27:45.165374 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 4/8] 10.222.0.52 () {58 vars in 1324 bytes} [Mon Aug 29 22:27:45 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 22:27:45.171338 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3866a0-8ff6-4a25-bd29-b20fbc9fd885 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:27:45.181119 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3866a0-8ff6-4a25-bd29-b20fbc9fd885 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:27:45.185578 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94c839f-f331-4714-8faa-84539584cedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:27:45.186198 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c839f-f331-4714-8faa-84539584cedf admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:45.186482 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c839f-f331-4714-8faa-84539584cedf admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:45.186760 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c839f-f331-4714-8faa-84539584cedf admin admin] PATH_INFO: `/v3/users/9cc82f35b5c947928c9caa5d0b7e11fc` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:45.188923 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94c839f-f331-4714-8faa-84539584cedf admin admin] RBAC: Authorizing `identity:get_user(user_id=9cc82f35b5c947928c9caa5d0b7e11fc)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:27:45.190780 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94c839f-f331-4714-8faa-84539584cedf admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:27:45.192301 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 5/9] 10.222.0.52 () {56 vars in 1198 bytes} [Mon Aug 29 22:27:45 2022] GET /identity/v3/users/9cc82f35b5c947928c9caa5d0b7e11fc => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:27:47.145684 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628efefa-8219-4e27-a04d-8692f74bfffc None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:47.146418 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628efefa-8219-4e27-a04d-8692f74bfffc None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:47.146418 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628efefa-8219-4e27-a04d-8692f74bfffc None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:47.147520 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 5/10] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:27:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:27:47.152584 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0180499d-8afa-4095-ba3f-67b5e0737f43 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:47.153092 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0180499d-8afa-4095-ba3f-67b5e0737f43 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:47.153258 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0180499d-8afa-4095-ba3f-67b5e0737f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:47.172144 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-0180499d-8afa-4095-ba3f-67b5e0737f43 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:27:47.173876 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-0180499d-8afa-4095-ba3f-67b5e0737f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:27:47.245278 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ce0b2-af8a-4963-bb93-5eaf14285c5e None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:47.245562 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ce0b2-af8a-4963-bb93-5eaf14285c5e None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:47.245842 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ce0b2-af8a-4963-bb93-5eaf14285c5e None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:47.247149 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 6/11] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:27:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:27:47.252797 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67238327-74cf-4a2e-a45f-0e7fc3a8f067 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:47.253080 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67238327-74cf-4a2e-a45f-0e7fc3a8f067 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:47.253352 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67238327-74cf-4a2e-a45f-0e7fc3a8f067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:47.256095 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 6/12] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:27:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:27:47.296474 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG passlib.handlers.bcrypt [None req-67238327-74cf-4a2e-a45f-0e7fc3a8f067 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=93271) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 29 22:27:47.297359 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG passlib.handlers.bcrypt [None req-67238327-74cf-4a2e-a45f-0e7fc3a8f067 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=93271) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 29 22:27:47.347589 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-67238327-74cf-4a2e-a45f-0e7fc3a8f067 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:27:47.349211 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-67238327-74cf-4a2e-a45f-0e7fc3a8f067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:27:47.449941 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 7/13] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:27:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 22:27:47.532173 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0180499d-8afa-4095-ba3f-67b5e0737f43 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:27:47.558101 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:27:47.639300 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0180499d-8afa-4095-ba3f-67b5e0737f43 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:27:47.674222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf16e1a-72f0-4f1e-8d6e-e75c8c7f3b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:27:47.674875 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf16e1a-72f0-4f1e-8d6e-e75c8c7f3b5f None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:47.675150 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf16e1a-72f0-4f1e-8d6e-e75c8c7f3b5f None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:47.675419 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf16e1a-72f0-4f1e-8d6e-e75c8c7f3b5f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:47.687315 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf16e1a-72f0-4f1e-8d6e-e75c8c7f3b5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:27:47.696205 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-eaf16e1a-72f0-4f1e-8d6e-e75c8c7f3b5f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:27:47.696205 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:27:47.696205 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:27:47.696205 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:27:47.696205 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:27:47.696205 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:27:47.696205 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:27:47.696205 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:27:47.696205 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:27:47.696205 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:27:47.696205 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:27:47.696205 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:27:47.696205 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:27:47.696205 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:27:47.696205 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:27:47.696205 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:27:47.697176 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:27:47.697176 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:27:47.697176 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:27:47.697176 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:27:47.697176 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:27:47.697176 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:27:47.697176 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:27:47.697176 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:27:47.697176 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:27:47.697176 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:27:47.697176 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:27:47.697176 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:27:47.697176 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:27:47.697176 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:27:47.697176 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:27:47.697176 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:27:47.697981 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:27:47.697981 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:27:47.697981 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:27:47.697981 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:27:47.697981 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:27:47.697981 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:27:47.697981 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:27:47.697981 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:27:47.697981 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:27:47.697981 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:27:47.697981 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:27:47.697981 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:27:47.697981 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 7/14] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Aug 29 22:27:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:27:47.703381 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67238327-74cf-4a2e-a45f-0e7fc3a8f067 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:27:47.717366 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf16e1a-72f0-4f1e-8d6e-e75c8c7f3b5f None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:27:47.800805 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:27:47.808731 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67238327-74cf-4a2e-a45f-0e7fc3a8f067 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:27:47.811111 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:27:47.818208 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf16e1a-72f0-4f1e-8d6e-e75c8c7f3b5f None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:27:47.847998 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f183d5c8-4dfd-447e-b2ae-77e7f328b4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:27:47.848919 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f183d5c8-4dfd-447e-b2ae-77e7f328b4a1 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:47.849205 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f183d5c8-4dfd-447e-b2ae-77e7f328b4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:47.849487 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f183d5c8-4dfd-447e-b2ae-77e7f328b4a1 None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:47.850327 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f183d5c8-4dfd-447e-b2ae-77e7f328b4a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:27:47.850626 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f183d5c8-4dfd-447e-b2ae-77e7f328b4a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:27:47.852050 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f183d5c8-4dfd-447e-b2ae-77e7f328b4a1 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:27:47.854121 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df70b3df-30d5-4fd1-b190-3d86b1f74425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:27:47.854715 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df70b3df-30d5-4fd1-b190-3d86b1f74425 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:47.854992 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df70b3df-30d5-4fd1-b190-3d86b1f74425 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:47.855262 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df70b3df-30d5-4fd1-b190-3d86b1f74425 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:47.865553 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 8/15] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:27:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:27:47.871053 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f183d5c8-4dfd-447e-b2ae-77e7f328b4a1 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:27:47.924671 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df70b3df-30d5-4fd1-b190-3d86b1f74425 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:27:47.926073 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df70b3df-30d5-4fd1-b190-3d86b1f74425 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:27:47.976911 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f183d5c8-4dfd-447e-b2ae-77e7f328b4a1 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:27:48.008831 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:27:48.014519 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42372ee0-dde5-4931-9868-b9b610f72be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:27:48.014779 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42372ee0-dde5-4931-9868-b9b610f72be7 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:48.019797 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42372ee0-dde5-4931-9868-b9b610f72be7 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:48.019797 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42372ee0-dde5-4931-9868-b9b610f72be7 None admin] PATH_INFO: `/v3/users/9cc82f35b5c947928c9caa5d0b7e11fc` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:48.019797 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42372ee0-dde5-4931-9868-b9b610f72be7 None admin] RBAC: Authorizing `identity:get_user(user_id=9cc82f35b5c947928c9caa5d0b7e11fc)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:27:48.019797 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42372ee0-dde5-4931-9868-b9b610f72be7 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:27:48.021090 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 9/16] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:27:47 2022] GET /identity/v3/users/9cc82f35b5c947928c9caa5d0b7e11fc => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:27:48.026552 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42372ee0-dde5-4931-9868-b9b610f72be7 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:27:48.028750 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 8/17] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:27:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 22:27:48.034749 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df70b3df-30d5-4fd1-b190-3d86b1f74425 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:27:48.063027 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:27:48.129085 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42372ee0-dde5-4931-9868-b9b610f72be7 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:27:48.135474 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df70b3df-30d5-4fd1-b190-3d86b1f74425 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:27:48.162891 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcce97e-2c62-4fb3-989a-4c093bd12fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:27:48.163518 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcce97e-2c62-4fb3-989a-4c093bd12fb0 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:48.163796 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcce97e-2c62-4fb3-989a-4c093bd12fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:48.164066 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcce97e-2c62-4fb3-989a-4c093bd12fb0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:48.165207 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcce97e-2c62-4fb3-989a-4c093bd12fb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:27:48.166649 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcce97e-2c62-4fb3-989a-4c093bd12fb0 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:27:48.168370 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 10/18] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:27:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:27:48.169170 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58f3d9-df70-4af3-943d-141ac3afd9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:27:48.169809 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58f3d9-df70-4af3-943d-141ac3afd9dd None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:48.170090 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58f3d9-df70-4af3-943d-141ac3afd9dd None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:48.170362 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58f3d9-df70-4af3-943d-141ac3afd9dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:48.175313 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcce97e-2c62-4fb3-989a-4c093bd12fb0 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:27:48.181573 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b58f3d9-df70-4af3-943d-141ac3afd9dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:27:48.183056 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b58f3d9-df70-4af3-943d-141ac3afd9dd None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:27:48.200489 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-9b58f3d9-df70-4af3-943d-141ac3afd9dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:27:48.200489 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:27:48.200489 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:27:48.200489 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:27:48.200489 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:27:48.200489 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:27:48.200489 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:27:48.200489 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:27:48.200489 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:27:48.200489 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:27:48.200489 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:27:48.200489 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:27:48.200489 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:27:48.200489 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:27:48.200489 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:27:48.200489 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:27:48.201399 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:27:48.201399 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:27:48.201399 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:27:48.201399 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:27:48.201399 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:27:48.201399 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:27:48.201399 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:27:48.201399 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:27:48.201399 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:27:48.201399 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:27:48.201399 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:27:48.201399 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:27:48.201399 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:27:48.201399 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:27:48.201399 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:27:48.201399 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:27:48.201399 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:27:48.202406 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:27:48.202406 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:27:48.202406 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:27:48.202406 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 9/19] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:27:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:27:48.207749 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58f3d9-df70-4af3-943d-141ac3afd9dd None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:27:48.242586 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:27:48.278036 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcce97e-2c62-4fb3-989a-4c093bd12fb0 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:27:48.279687 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:27:48.311670 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58f3d9-df70-4af3-943d-141ac3afd9dd None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:27:48.313762 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172cc006-aa6d-4005-ad7a-bf3476302773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:27:48.314735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172cc006-aa6d-4005-ad7a-bf3476302773 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:48.315240 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172cc006-aa6d-4005-ad7a-bf3476302773 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:48.315825 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172cc006-aa6d-4005-ad7a-bf3476302773 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:48.316641 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172cc006-aa6d-4005-ad7a-bf3476302773 None admin] RBAC: Adding query filter params (role.id=882f35bd847a406eb2222aeafc517a46, scope.domain.id=default, user.id=9cc82f35b5c947928c9caa5d0b7e11fc) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:27:48.317050 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172cc006-aa6d-4005-ad7a-bf3476302773 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:27:48.318508 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172cc006-aa6d-4005-ad7a-bf3476302773 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:27:48.326703 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 11/20] 10.222.0.52 () {56 vars in 1322 bytes} [Mon Aug 29 22:27:48 2022] GET /identity/v3/role_assignments?user.id=9cc82f35b5c947928c9caa5d0b7e11fc&scope.domain.id=default&role.id=882f35bd847a406eb2222aeafc517a46 => generated 233 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:27:48.348204 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66171849-4ea8-49d8-9b89-0b6d3c8f1c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:27:48.348901 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66171849-4ea8-49d8-9b89-0b6d3c8f1c11 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:48.349189 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66171849-4ea8-49d8-9b89-0b6d3c8f1c11 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:48.349467 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66171849-4ea8-49d8-9b89-0b6d3c8f1c11 None admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:48.350143 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66171849-4ea8-49d8-9b89-0b6d3c8f1c11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:27:48.350451 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66171849-4ea8-49d8-9b89-0b6d3c8f1c11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:27:48.351918 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66171849-4ea8-49d8-9b89-0b6d3c8f1c11 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:27:48.376624 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 10/21] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:27:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:27:50.348912 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc42450-4a4e-40c9-9548-0c5ed1516279 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:50.349920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc42450-4a4e-40c9-9548-0c5ed1516279 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:50.350428 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc42450-4a4e-40c9-9548-0c5ed1516279 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:50.351943 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 12/22] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:27:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:27:50.357414 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142eaa93-554a-425d-849e-cb81d669af87 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:50.357810 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142eaa93-554a-425d-849e-cb81d669af87 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:50.357952 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142eaa93-554a-425d-849e-cb81d669af87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:50.381753 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-142eaa93-554a-425d-849e-cb81d669af87 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:27:50.383229 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-142eaa93-554a-425d-849e-cb81d669af87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:27:50.474583 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 11/23] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:27:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:27:50.623256 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511cfc9b-070a-4b21-83f5-467b9a75b296 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:50.623581 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511cfc9b-070a-4b21-83f5-467b9a75b296 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:50.623760 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511cfc9b-070a-4b21-83f5-467b9a75b296 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:50.625120 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 13/24] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:27:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:27:50.630584 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07ab03-4ee7-4e6a-9a4a-cfa1e8700340 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:50.631122 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07ab03-4ee7-4e6a-9a4a-cfa1e8700340 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:50.631620 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07ab03-4ee7-4e6a-9a4a-cfa1e8700340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:50.658173 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-4d07ab03-4ee7-4e6a-9a4a-cfa1e8700340 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:27:50.659703 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-4d07ab03-4ee7-4e6a-9a4a-cfa1e8700340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:27:50.685483 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:27:50.741664 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 12/25] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:27:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:27:50.975846 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511cfc9b-070a-4b21-83f5-467b9a75b296 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:27:51.108493 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511cfc9b-070a-4b21-83f5-467b9a75b296 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:27:51.161205 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d69b45-1829-4a4a-9f86-0f66d5eb8417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:27:51.161857 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d69b45-1829-4a4a-9f86-0f66d5eb8417 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:51.162136 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d69b45-1829-4a4a-9f86-0f66d5eb8417 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:51.162406 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d69b45-1829-4a4a-9f86-0f66d5eb8417 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:51.181616 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d69b45-1829-4a4a-9f86-0f66d5eb8417 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:27:51.190888 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-a7d69b45-1829-4a4a-9f86-0f66d5eb8417 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:27:51.190888 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:27:51.190888 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:27:51.190888 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:27:51.190888 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:27:51.190888 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:27:51.190888 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:27:51.190888 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:27:51.190888 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:27:51.190888 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:27:51.190888 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:27:51.190888 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:27:51.190888 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:27:51.190888 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:27:51.190888 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:27:51.190888 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:27:51.191723 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:27:51.191723 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:27:51.191723 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:27:51.191723 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:27:51.191723 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:27:51.191723 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:27:51.191723 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:27:51.191723 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:27:51.191723 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:27:51.191723 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:27:51.191723 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:27:51.191723 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:27:51.191723 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:27:51.191723 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:27:51.191723 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:27:51.191723 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:27:51.192474 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:27:51.192474 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:27:51.192474 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:27:51.192474 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:27:51.192474 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:27:51.192474 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:27:51.192474 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:27:51.192474 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:27:51.192474 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:27:51.192474 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:27:51.192474 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:27:51.192474 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:27:51.192474 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 14/26] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Aug 29 22:27:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:27:51.198066 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d07ab03-4ee7-4e6a-9a4a-cfa1e8700340 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:27:51.238106 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79ccc4-260b-42ea-bcf1-0a3daf8f4650 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:51.238389 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79ccc4-260b-42ea-bcf1-0a3daf8f4650 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:51.238584 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79ccc4-260b-42ea-bcf1-0a3daf8f4650 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:51.240218 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 15/27] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:27:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:27:51.247398 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2791e8-4ea2-40c7-bc64-9e7eb7aced66 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:51.247687 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2791e8-4ea2-40c7-bc64-9e7eb7aced66 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:51.247959 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2791e8-4ea2-40c7-bc64-9e7eb7aced66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:51.295451 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-9b2791e8-4ea2-40c7-bc64-9e7eb7aced66 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:27:51.301347 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-9b2791e8-4ea2-40c7-bc64-9e7eb7aced66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:27:51.344662 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:27:51.412341 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d07ab03-4ee7-4e6a-9a4a-cfa1e8700340 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:27:51.448025 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 16/28] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:27:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:27:51.448983 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2791e8-4ea2-40c7-bc64-9e7eb7aced66 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:27:51.534141 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90814503-67e9-4cac-a61c-9a021a6b3136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:27:51.534825 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90814503-67e9-4cac-a61c-9a021a6b3136 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:51.535169 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90814503-67e9-4cac-a61c-9a021a6b3136 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:51.535511 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90814503-67e9-4cac-a61c-9a021a6b3136 None admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:51.536258 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90814503-67e9-4cac-a61c-9a021a6b3136 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:27:51.536625 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90814503-67e9-4cac-a61c-9a021a6b3136 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:27:51.538116 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90814503-67e9-4cac-a61c-9a021a6b3136 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:27:51.574987 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 13/29] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:27:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:27:51.581171 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90814503-67e9-4cac-a61c-9a021a6b3136 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:27:51.625496 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2791e8-4ea2-40c7-bc64-9e7eb7aced66 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:27:51.633067 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:27:51.689673 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc720a2-c32f-429a-a8de-ade1b969c4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:27:51.690314 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc720a2-c32f-429a-a8de-ade1b969c4d4 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:51.690593 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc720a2-c32f-429a-a8de-ade1b969c4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:51.690868 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc720a2-c32f-429a-a8de-ade1b969c4d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:51.730017 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc720a2-c32f-429a-a8de-ade1b969c4d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:27:51.731509 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc720a2-c32f-429a-a8de-ade1b969c4d4 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:27:51.768045 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-6fc720a2-c32f-429a-a8de-ade1b969c4d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:27:51.768045 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:27:51.768045 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:27:51.768045 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:27:51.768045 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:27:51.768045 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:27:51.768045 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:27:51.768045 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:27:51.768045 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:27:51.768045 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:27:51.768045 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:27:51.768045 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:27:51.768045 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:27:51.768045 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:27:51.768045 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:27:51.768045 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:27:51.768987 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:27:51.768987 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:27:51.768987 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:27:51.768987 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:27:51.768987 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:27:51.768987 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:27:51.768987 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:27:51.768987 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:27:51.768987 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:27:51.768987 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:27:51.768987 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:27:51.768987 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:27:51.768987 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:27:51.768987 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:27:51.768987 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:27:51.768987 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:27:51.768987 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:27:51.769786 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:27:51.769786 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:27:51.769786 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:27:51.771644 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 17/30] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:27:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:27:51.773256 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb94dd7-7ce8-44ff-aa9c-fe84d44f9c72 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:51.773649 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb94dd7-7ce8-44ff-aa9c-fe84d44f9c72 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:51.774036 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb94dd7-7ce8-44ff-aa9c-fe84d44f9c72 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:51.775536 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 18/31] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:27:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:27:51.776615 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb94dd7-7ce8-44ff-aa9c-fe84d44f9c72 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:27:51.788252 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90814503-67e9-4cac-a61c-9a021a6b3136 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:27:51.793911 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:27:51.826001 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4882c28-1179-4389-8a73-30c862c15c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:27:51.826749 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4882c28-1179-4389-8a73-30c862c15c5d None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:51.827075 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4882c28-1179-4389-8a73-30c862c15c5d None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:51.827460 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4882c28-1179-4389-8a73-30c862c15c5d None admin] PATH_INFO: `/v3/users/9cc82f35b5c947928c9caa5d0b7e11fc` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:51.829090 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4882c28-1179-4389-8a73-30c862c15c5d None admin] RBAC: Authorizing `identity:get_user(user_id=9cc82f35b5c947928c9caa5d0b7e11fc)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:27:51.830602 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4882c28-1179-4389-8a73-30c862c15c5d None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:27:51.832284 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 14/32] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:27:51 2022] GET /identity/v3/users/9cc82f35b5c947928c9caa5d0b7e11fc => generated 247 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:27:51.833365 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4882c28-1179-4389-8a73-30c862c15c5d None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:27:51.895200 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb94dd7-7ce8-44ff-aa9c-fe84d44f9c72 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:27:51.924644 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:27:51.932419 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed95bfa2-8a5c-4526-a575-201fffca6519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:27:51.933247 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95bfa2-8a5c-4526-a575-201fffca6519 None admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:51.933693 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95bfa2-8a5c-4526-a575-201fffca6519 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:51.934134 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95bfa2-8a5c-4526-a575-201fffca6519 None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:51.936949 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed95bfa2-8a5c-4526-a575-201fffca6519 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:27:51.936949 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed95bfa2-8a5c-4526-a575-201fffca6519 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:27:51.943059 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4882c28-1179-4389-8a73-30c862c15c5d None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:27:51.990468 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b56db0-5e78-4971-a7a3-924f6c356b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:27:51.991078 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b56db0-5e78-4971-a7a3-924f6c356b4e None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:51.991354 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b56db0-5e78-4971-a7a3-924f6c356b4e None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:51.991622 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b56db0-5e78-4971-a7a3-924f6c356b4e None admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:51.992262 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b56db0-5e78-4971-a7a3-924f6c356b4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:27:51.992560 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b56db0-5e78-4971-a7a3-924f6c356b4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:27:51.993951 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b56db0-5e78-4971-a7a3-924f6c356b4e None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:27:52.006706 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 15/33] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:27:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:27:52.007923 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f3bee-b648-4da5-9722-798f09fec133 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:52.008240 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f3bee-b648-4da5-9722-798f09fec133 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:52.008481 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f3bee-b648-4da5-9722-798f09fec133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:52.026590 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-2c3f3bee-b648-4da5-9722-798f09fec133 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:27:52.027768 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-2c3f3bee-b648-4da5-9722-798f09fec133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:27:52.118211 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 16/34] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:27:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:27:52.118974 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3f3bee-b648-4da5-9722-798f09fec133 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:27:52.154730 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 19/35] 10.222.0.52 () {60 vars in 1131 bytes} [Mon Aug 29 22:27:51 2022] POST /identity/v3/roles => generated 219 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:27:52.155483 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed95bfa2-8a5c-4526-a575-201fffca6519 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:27:52.210821 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:27:52.239250 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3f3bee-b648-4da5-9722-798f09fec133 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:27:52.262678 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:27:52.283651 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0028db-0eb7-408b-b848-517e96f6107d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:27:52.284406 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0028db-0eb7-408b-b848-517e96f6107d None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:52.284791 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0028db-0eb7-408b-b848-517e96f6107d None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:52.285231 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0028db-0eb7-408b-b848-517e96f6107d None admin] PATH_INFO: `/v3/domains/default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:52.287415 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0028db-0eb7-408b-b848-517e96f6107d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:27:52.289674 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0028db-0eb7-408b-b848-517e96f6107d None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:27:52.291907 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 17/36] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:27:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:27:52.294545 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2dccd5-cb4b-446e-926a-3565cc745819 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:52.294711 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2dccd5-cb4b-446e-926a-3565cc745819 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:52.295270 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2dccd5-cb4b-446e-926a-3565cc745819 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:52.297996 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 18/37] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:27:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:27:52.299308 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2236f9-2dd1-4d56-95db-8b5b87064a6e None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:52.299620 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2236f9-2dd1-4d56-95db-8b5b87064a6e None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:52.304945 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2236f9-2dd1-4d56-95db-8b5b87064a6e None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:52.304945 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 19/38] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:27:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:27:52.304945 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2236f9-2dd1-4d56-95db-8b5b87064a6e None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:27:52.320468 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed95bfa2-8a5c-4526-a575-201fffca6519 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:27:52.368579 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b97572-2ea6-4834-a4a5-ed38cd08888a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:27:52.369270 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b97572-2ea6-4834-a4a5-ed38cd08888a None admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:52.369629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b97572-2ea6-4834-a4a5-ed38cd08888a None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:52.369981 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b97572-2ea6-4834-a4a5-ed38cd08888a None admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:52.370712 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b97572-2ea6-4834-a4a5-ed38cd08888a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:27:52.372377 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b97572-2ea6-4834-a4a5-ed38cd08888a None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:27:52.436317 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2236f9-2dd1-4d56-95db-8b5b87064a6e None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:27:52.470414 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9700fc4f-fc31-40de-9551-9754b984490d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:27:52.471057 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700fc4f-fc31-40de-9551-9754b984490d None admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:52.471332 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700fc4f-fc31-40de-9551-9754b984490d None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:52.471600 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700fc4f-fc31-40de-9551-9754b984490d None admin] PATH_INFO: `/v3/domains/default/users/9cc82f35b5c947928c9caa5d0b7e11fc/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:52.473536 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9700fc4f-fc31-40de-9551-9754b984490d None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=9cc82f35b5c947928c9caa5d0b7e11fc, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:27:52.474973 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9700fc4f-fc31-40de-9551-9754b984490d None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:27:52.489230 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 20/39] 10.222.0.52 () {60 vars in 1141 bytes} [Mon Aug 29 22:27:52 2022] POST /identity/v3/projects => generated 301 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:27:52.494161 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cce77a-d985-49e7-bc99-5f0f7a5f7637 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:52.494161 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cce77a-d985-49e7-bc99-5f0f7a5f7637 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:52.494161 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cce77a-d985-49e7-bc99-5f0f7a5f7637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:52.512173 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-16cce77a-d985-49e7-bc99-5f0f7a5f7637 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:27:52.513867 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-16cce77a-d985-49e7-bc99-5f0f7a5f7637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:27:52.601899 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 21/40] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:27:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:27:52.601899 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff0e7e-ca35-4804-90ef-7f2b95cd806e None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:52.601899 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff0e7e-ca35-4804-90ef-7f2b95cd806e None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:52.601899 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff0e7e-ca35-4804-90ef-7f2b95cd806e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:52.620340 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-1cff0e7e-ca35-4804-90ef-7f2b95cd806e None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:27:52.622296 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-1cff0e7e-ca35-4804-90ef-7f2b95cd806e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:27:52.674604 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 20/41] 10.222.0.52 () {58 vars in 1361 bytes} [Mon Aug 29 22:27:52 2022] PUT /identity/v3/domains/default/users/9cc82f35b5c947928c9caa5d0b7e11fc/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:27:52.675870 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9700fc4f-fc31-40de-9551-9754b984490d None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:27:52.688280 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:27:52.714100 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 22/42] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:27:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:27:52.793553 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:27:52.913102 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9700fc4f-fc31-40de-9551-9754b984490d None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:27:52.918282 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff0e7e-ca35-4804-90ef-7f2b95cd806e None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:27:52.955379 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6120d0ff-11a0-4c25-a9b8-30d96ac407e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:27:52.956001 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120d0ff-11a0-4c25-a9b8-30d96ac407e0 None admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:52.956289 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120d0ff-11a0-4c25-a9b8-30d96ac407e0 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:52.956906 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120d0ff-11a0-4c25-a9b8-30d96ac407e0 None admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:52.957615 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6120d0ff-11a0-4c25-a9b8-30d96ac407e0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:27:52.960229 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6120d0ff-11a0-4c25-a9b8-30d96ac407e0 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:27:53.030458 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff0e7e-ca35-4804-90ef-7f2b95cd806e None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:27:53.068444 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f29dd7-f4fb-4ce0-a99f-16235967de63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:27:53.069095 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f29dd7-f4fb-4ce0-a99f-16235967de63 None admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:53.069413 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f29dd7-f4fb-4ce0-a99f-16235967de63 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:53.069651 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f29dd7-f4fb-4ce0-a99f-16235967de63 None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:53.070319 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f29dd7-f4fb-4ce0-a99f-16235967de63 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:27:53.071705 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f29dd7-f4fb-4ce0-a99f-16235967de63 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:27:53.080919 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 21/43] 10.222.0.52 () {60 vars in 1131 bytes} [Mon Aug 29 22:27:52 2022] POST /identity/v3/roles => generated 225 bytes in 398 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:27:53.080919 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6120d0ff-11a0-4c25-a9b8-30d96ac407e0 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:27:53.158853 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:27:53.177045 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 23/44] 10.222.0.52 () {60 vars in 1131 bytes} [Mon Aug 29 22:27:52 2022] POST /identity/v3/roles => generated 223 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:27:53.221234 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6120d0ff-11a0-4c25-a9b8-30d96ac407e0 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:27:53.264088 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cf8013-cee2-4e4a-bd02-0869185de8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:27:53.264942 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf8013-cee2-4e4a-bd02-0869185de8d4 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:53.265226 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf8013-cee2-4e4a-bd02-0869185de8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:53.266018 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf8013-cee2-4e4a-bd02-0869185de8d4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:53.267596 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cf8013-cee2-4e4a-bd02-0869185de8d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:27:53.269121 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cf8013-cee2-4e4a-bd02-0869185de8d4 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:27:53.271093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 22/45] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:27:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:27:53.276921 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f29dd7-f4fb-4ce0-a99f-16235967de63 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:27:53.329273 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:27:53.399643 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f29dd7-f4fb-4ce0-a99f-16235967de63 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:27:53.434924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ad2b28-bd54-45a7-8c0d-c6573e032332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:27:53.435465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad2b28-bd54-45a7-8c0d-c6573e032332 None admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:53.435749 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad2b28-bd54-45a7-8c0d-c6573e032332 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:53.436025 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad2b28-bd54-45a7-8c0d-c6573e032332 None admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:53.436680 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ad2b28-bd54-45a7-8c0d-c6573e032332 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:27:53.438785 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ad2b28-bd54-45a7-8c0d-c6573e032332 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:27:53.524751 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 24/46] 10.222.0.52 () {60 vars in 1141 bytes} [Mon Aug 29 22:27:53 2022] POST /identity/v3/projects => generated 312 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:27:54.714256 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890183eb-0c92-40ed-920b-893e246224a4 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:54.714760 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890183eb-0c92-40ed-920b-893e246224a4 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:54.714843 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890183eb-0c92-40ed-920b-893e246224a4 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:54.716189 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 23/47] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:27:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:27:54.722210 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d5cac-94f8-4126-bfdd-f6a4c4ea58d9 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:54.722630 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d5cac-94f8-4126-bfdd-f6a4c4ea58d9 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:54.722772 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d5cac-94f8-4126-bfdd-f6a4c4ea58d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:54.740778 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-9b9d5cac-94f8-4126-bfdd-f6a4c4ea58d9 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:27:54.741986 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-9b9d5cac-94f8-4126-bfdd-f6a4c4ea58d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:27:54.808049 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 25/48] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:27:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:27:55.079519 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890183eb-0c92-40ed-920b-893e246224a4 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:27:55.181092 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890183eb-0c92-40ed-920b-893e246224a4 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:27:55.210657 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:27:55.216207 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5419c966-981d-40cc-94e2-94965285e02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:27:55.216885 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5419c966-981d-40cc-94e2-94965285e02e None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:55.217171 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5419c966-981d-40cc-94e2-94965285e02e None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:55.217453 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5419c966-981d-40cc-94e2-94965285e02e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:55.227853 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5419c966-981d-40cc-94e2-94965285e02e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:27:55.233186 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-5419c966-981d-40cc-94e2-94965285e02e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:27:55.233186 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:27:55.233186 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:27:55.233186 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:27:55.233186 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:27:55.233186 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:27:55.233186 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:27:55.233186 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:27:55.233186 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:27:55.233186 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:27:55.233186 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:27:55.233186 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:27:55.233186 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:27:55.233186 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:27:55.233186 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:27:55.233186 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:27:55.234006 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:27:55.234006 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:27:55.234006 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:27:55.234006 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:27:55.234006 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:27:55.234006 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:27:55.234006 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:27:55.234006 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:27:55.234006 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:27:55.234006 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:27:55.234006 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:27:55.234006 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:27:55.234006 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:27:55.234006 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:27:55.234006 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:27:55.234006 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:27:55.234925 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:27:55.234925 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:27:55.234925 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:27:55.234925 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:27:55.234925 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:27:55.234925 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:27:55.234925 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:27:55.234925 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:27:55.234925 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:27:55.234925 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:27:55.234925 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:27:55.234925 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:27:55.234925 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 24/49] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Aug 29 22:27:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:27:55.240068 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9d5cac-94f8-4126-bfdd-f6a4c4ea58d9 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:27:55.264119 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:27:55.347714 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9d5cac-94f8-4126-bfdd-f6a4c4ea58d9 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:27:55.382392 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d1264e-cbed-4c1d-b03f-aed3fe624fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:27:55.383000 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1264e-cbed-4c1d-b03f-aed3fe624fb9 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:55.383281 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1264e-cbed-4c1d-b03f-aed3fe624fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:55.383556 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1264e-cbed-4c1d-b03f-aed3fe624fb9 None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:55.384223 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d1264e-cbed-4c1d-b03f-aed3fe624fb9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:27:55.384528 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d1264e-cbed-4c1d-b03f-aed3fe624fb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:27:55.385954 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d1264e-cbed-4c1d-b03f-aed3fe624fb9 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:27:55.394254 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 26/50] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:27:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:27:55.400240 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5419c966-981d-40cc-94e2-94965285e02e None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:27:55.463217 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f90fc-d2dc-46bf-89dd-ac773faa66a4 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:55.463511 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f90fc-d2dc-46bf-89dd-ac773faa66a4 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:55.463798 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f90fc-d2dc-46bf-89dd-ac773faa66a4 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:55.465173 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 27/51] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:27:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:27:55.470550 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de66e30-d4a9-4618-8995-242e67c15fc6 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:55.470839 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de66e30-d4a9-4618-8995-242e67c15fc6 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:55.471112 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de66e30-d4a9-4618-8995-242e67c15fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:55.491833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-9de66e30-d4a9-4618-8995-242e67c15fc6 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:27:55.493079 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-9de66e30-d4a9-4618-8995-242e67c15fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:27:55.503307 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5419c966-981d-40cc-94e2-94965285e02e None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:27:55.537402 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4816f75f-3af6-41b6-a8d9-cdddf5f81885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:27:55.537852 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4816f75f-3af6-41b6-a8d9-cdddf5f81885 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:55.538141 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4816f75f-3af6-41b6-a8d9-cdddf5f81885 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:55.538420 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4816f75f-3af6-41b6-a8d9-cdddf5f81885 None admin] PATH_INFO: `/v3/users/9cc82f35b5c947928c9caa5d0b7e11fc` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:55.539552 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4816f75f-3af6-41b6-a8d9-cdddf5f81885 None admin] RBAC: Authorizing `identity:get_user(user_id=9cc82f35b5c947928c9caa5d0b7e11fc)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:27:55.541053 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4816f75f-3af6-41b6-a8d9-cdddf5f81885 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:27:55.542798 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 25/52] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:27:55 2022] GET /identity/v3/users/9cc82f35b5c947928c9caa5d0b7e11fc => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:27:55.549199 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4816f75f-3af6-41b6-a8d9-cdddf5f81885 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:27:55.566101 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 28/53] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:27:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 22:27:55.612360 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:27:55.659705 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4816f75f-3af6-41b6-a8d9-cdddf5f81885 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:27:55.694619 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420867ce-000d-4448-a17d-6b252601af38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:27:55.695280 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420867ce-000d-4448-a17d-6b252601af38 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:55.695568 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420867ce-000d-4448-a17d-6b252601af38 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:55.695848 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420867ce-000d-4448-a17d-6b252601af38 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:55.697011 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420867ce-000d-4448-a17d-6b252601af38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:27:55.698469 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420867ce-000d-4448-a17d-6b252601af38 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:27:55.700036 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 26/54] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:27:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:27:55.706152 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de66e30-d4a9-4618-8995-242e67c15fc6 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:27:55.721625 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:27:55.807298 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de66e30-d4a9-4618-8995-242e67c15fc6 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:27:55.831584 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420867ce-000d-4448-a17d-6b252601af38 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:27:55.843488 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3e928-ea53-4f9f-baad-b679196783e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:27:55.844781 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3e928-ea53-4f9f-baad-b679196783e1 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:55.845035 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3e928-ea53-4f9f-baad-b679196783e1 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:55.845305 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3e928-ea53-4f9f-baad-b679196783e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:55.846026 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de3e928-ea53-4f9f-baad-b679196783e1 None admin] RBAC: Adding query filter params (role.id=882f35bd847a406eb2222aeafc517a46, scope.domain.id=default, user.id=9cc82f35b5c947928c9caa5d0b7e11fc) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:27:55.846331 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de3e928-ea53-4f9f-baad-b679196783e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:27:55.847746 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de3e928-ea53-4f9f-baad-b679196783e1 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:27:55.853685 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 29/55] 10.222.0.52 () {56 vars in 1322 bytes} [Mon Aug 29 22:27:55 2022] GET /identity/v3/role_assignments?user.id=9cc82f35b5c947928c9caa5d0b7e11fc&scope.domain.id=default&role.id=882f35bd847a406eb2222aeafc517a46 => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:27:55.946513 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420867ce-000d-4448-a17d-6b252601af38 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:27:55.979649 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:27:55.989823 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d757ca2a-aa26-47ea-a0f9-c8f0933e8341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:27:55.991077 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d757ca2a-aa26-47ea-a0f9-c8f0933e8341 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:55.991077 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d757ca2a-aa26-47ea-a0f9-c8f0933e8341 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:55.991077 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d757ca2a-aa26-47ea-a0f9-c8f0933e8341 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:55.992091 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d757ca2a-aa26-47ea-a0f9-c8f0933e8341 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:27:55.993623 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d757ca2a-aa26-47ea-a0f9-c8f0933e8341 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:27:55.995372 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 27/56] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:27:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:27:56.001322 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3e928-ea53-4f9f-baad-b679196783e1 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:27:56.191388 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:27:56.212132 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3e928-ea53-4f9f-baad-b679196783e1 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:27:56.246327 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621f4a88-38bb-42f0-8948-a016405bb430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:27:56.246939 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f4a88-38bb-42f0-8948-a016405bb430 None admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:56.247224 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f4a88-38bb-42f0-8948-a016405bb430 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:56.247520 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f4a88-38bb-42f0-8948-a016405bb430 None admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:56.248146 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621f4a88-38bb-42f0-8948-a016405bb430 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:27:56.249624 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621f4a88-38bb-42f0-8948-a016405bb430 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:27:56.337820 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 30/57] 10.222.0.52 () {60 vars in 1140 bytes} [Mon Aug 29 22:27:55 2022] POST /identity/v3/projects => generated 298 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:27:58.278623 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da01069-fa2b-4301-a541-04a439950830 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:58.279174 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da01069-fa2b-4301-a541-04a439950830 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:58.279174 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da01069-fa2b-4301-a541-04a439950830 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:58.280500 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 28/58] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:27:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:27:58.286386 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47849c31-da91-45c0-b3c1-2e678a35e7b6 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:58.286771 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47849c31-da91-45c0-b3c1-2e678a35e7b6 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:58.286949 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47849c31-da91-45c0-b3c1-2e678a35e7b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:58.306013 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-47849c31-da91-45c0-b3c1-2e678a35e7b6 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:27:58.307212 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-47849c31-da91-45c0-b3c1-2e678a35e7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:27:58.377517 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 31/59] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:27:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:27:58.649752 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da01069-fa2b-4301-a541-04a439950830 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:27:58.755233 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da01069-fa2b-4301-a541-04a439950830 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:27:58.790159 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ad9550-18b4-487a-a7e0-9dc8afa8a8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:27:58.790714 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad9550-18b4-487a-a7e0-9dc8afa8a8b5 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:58.790997 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad9550-18b4-487a-a7e0-9dc8afa8a8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:58.791271 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad9550-18b4-487a-a7e0-9dc8afa8a8b5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:58.792382 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ad9550-18b4-487a-a7e0-9dc8afa8a8b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:27:58.793870 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ad9550-18b4-487a-a7e0-9dc8afa8a8b5 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:27:58.795426 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 29/60] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:27:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:27:58.801477 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47849c31-da91-45c0-b3c1-2e678a35e7b6 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:27:58.860052 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:27:58.900084 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47849c31-da91-45c0-b3c1-2e678a35e7b6 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:27:58.935019 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79367aec-c42f-4899-a155-2118daae7f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:27:58.935695 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79367aec-c42f-4899-a155-2118daae7f59 None admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:58.935982 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79367aec-c42f-4899-a155-2118daae7f59 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:58.936263 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79367aec-c42f-4899-a155-2118daae7f59 None admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:58.936969 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79367aec-c42f-4899-a155-2118daae7f59 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:27:58.939139 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79367aec-c42f-4899-a155-2118daae7f59 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:27:59.053288 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 32/61] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Aug 29 22:27:58 2022] POST /identity/v3/users => generated 275 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:28:01.949524 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6583a-cd7f-46d7-80ed-a9353f99f23d None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:01.949524 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6583a-cd7f-46d7-80ed-a9353f99f23d None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:01.949524 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6583a-cd7f-46d7-80ed-a9353f99f23d None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:01.949524 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 30/62] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:28:01.952304 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1801c4ca-9de4-421b-b74d-ab33825ceb0d None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:01.953233 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1801c4ca-9de4-421b-b74d-ab33825ceb0d None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:01.953233 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1801c4ca-9de4-421b-b74d-ab33825ceb0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:01.985773 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-1801c4ca-9de4-421b-b74d-ab33825ceb0d None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:01.989060 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-1801c4ca-9de4-421b-b74d-ab33825ceb0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:02.098531 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 33/63] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:02.301687 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2273c7-b63d-4df8-8189-afa7a5aa51fa None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:02.302020 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2273c7-b63d-4df8-8189-afa7a5aa51fa None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:02.302195 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2273c7-b63d-4df8-8189-afa7a5aa51fa None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:02.303744 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 31/64] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:02.309183 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909a7ff-4e71-4385-8827-b7cb0feb5a5e None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:02.309478 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909a7ff-4e71-4385-8827-b7cb0feb5a5e None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:02.309762 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909a7ff-4e71-4385-8827-b7cb0feb5a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:02.334022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-3909a7ff-4e71-4385-8827-b7cb0feb5a5e None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:02.335218 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-3909a7ff-4e71-4385-8827-b7cb0feb5a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:02.404767 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:02.437804 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 34/65] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 22:28:02.490176 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2273c7-b63d-4df8-8189-afa7a5aa51fa None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:02.561745 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:02.606842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2273c7-b63d-4df8-8189-afa7a5aa51fa None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:02.618271 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cc9ebd-44d8-4f49-ac5d-4c545d8f16a1 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:02.618564 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cc9ebd-44d8-4f49-ac5d-4c545d8f16a1 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:02.618829 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cc9ebd-44d8-4f49-ac5d-4c545d8f16a1 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:02.620179 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 35/66] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:28:02.636128 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c84790-6f5b-4fe8-9bd3-052ba278cab0 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:02.636774 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c84790-6f5b-4fe8-9bd3-052ba278cab0 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:02.637276 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c84790-6f5b-4fe8-9bd3-052ba278cab0 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:02.638830 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 36/67] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:28:02.640065 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988df30-bcef-4335-b870-a4ca0691c788 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:02.640549 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988df30-bcef-4335-b870-a4ca0691c788 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:02.641050 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988df30-bcef-4335-b870-a4ca0691c788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:02.648366 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4d6fb9-061b-4b18-80bd-5863dfc876e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:02.648366 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d6fb9-061b-4b18-80bd-5863dfc876e5 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:02.648366 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d6fb9-061b-4b18-80bd-5863dfc876e5 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:02.648366 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d6fb9-061b-4b18-80bd-5863dfc876e5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:02.659774 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4d6fb9-061b-4b18-80bd-5863dfc876e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:02.664992 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-0988df30-bcef-4335-b870-a4ca0691c788 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:02.665380 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-3e4d6fb9-061b-4b18-80bd-5863dfc876e5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:28:02.665380 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:02.665380 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:02.665380 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:02.665380 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:02.665380 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:02.665380 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:02.665380 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:02.665380 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:02.665380 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:02.665380 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:02.665380 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:02.665380 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:02.665380 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:02.665380 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:02.665380 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:02.666191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:02.666191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:02.666191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:02.666191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:02.666191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:02.666191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:02.666191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:02.666191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:02.666191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:02.666191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:02.666191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:02.666191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:02.666191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:02.666191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:02.666191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:02.666191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:02.666934 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:02.666934 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:02.666934 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:02.666934 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:02.666934 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:02.666934 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:02.666934 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:02.666934 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:02.666934 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:02.666934 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:02.666934 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:28:02.666934 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:28:02.666934 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 32/68] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:28:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:02.669250 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-0988df30-bcef-4335-b870-a4ca0691c788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:02.673380 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5007bf1-9105-4f0e-9cb0-32d64156e951 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:02.673380 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5007bf1-9105-4f0e-9cb0-32d64156e951 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:02.673380 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5007bf1-9105-4f0e-9cb0-32d64156e951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:02.696370 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-e5007bf1-9105-4f0e-9cb0-32d64156e951 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:02.698447 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-e5007bf1-9105-4f0e-9cb0-32d64156e951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:02.744950 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 37/69] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:02.746247 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0988df30-bcef-4335-b870-a4ca0691c788 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:02.791582 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 33/70] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:02.792769 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5007bf1-9105-4f0e-9cb0-32d64156e951 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:02.905473 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:02.918175 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0988df30-bcef-4335-b870-a4ca0691c788 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:02.931093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:02.988210 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9882c3b7-640b-4ea9-8a85-39bcfc656f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:02.989659 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9882c3b7-640b-4ea9-8a85-39bcfc656f8b None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:02.990244 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9882c3b7-640b-4ea9-8a85-39bcfc656f8b None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:02.990734 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9882c3b7-640b-4ea9-8a85-39bcfc656f8b None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:02.991606 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9882c3b7-640b-4ea9-8a85-39bcfc656f8b None admin] RBAC: Adding query filter params (name=member) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:02.992123 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9882c3b7-640b-4ea9-8a85-39bcfc656f8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:02.994802 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9882c3b7-640b-4ea9-8a85-39bcfc656f8b None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:03.006355 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5007bf1-9105-4f0e-9cb0-32d64156e951 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:03.007833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 38/71] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Aug 29 22:28:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:03.013810 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d17ce24-fda6-4203-b60f-409c3e1cab13 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:03.014315 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d17ce24-fda6-4203-b60f-409c3e1cab13 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:03.014793 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d17ce24-fda6-4203-b60f-409c3e1cab13 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:03.016464 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 39/72] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:03.018045 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d17ce24-fda6-4203-b60f-409c3e1cab13 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:03.051988 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b594cbf-39cc-45df-9508-36a19484b0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:03.052818 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b594cbf-39cc-45df-9508-36a19484b0ab None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:03.053106 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b594cbf-39cc-45df-9508-36a19484b0ab None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:03.053706 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b594cbf-39cc-45df-9508-36a19484b0ab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:03.067060 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b594cbf-39cc-45df-9508-36a19484b0ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:03.071997 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-5b594cbf-39cc-45df-9508-36a19484b0ab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:28:03.071997 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:03.071997 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:03.071997 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:03.071997 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:03.071997 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:03.071997 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:03.071997 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:03.071997 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:03.071997 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:03.071997 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:03.071997 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:03.071997 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:03.071997 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:03.071997 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:03.071997 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:03.072914 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:03.072914 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:03.072914 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:03.072914 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:03.072914 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:03.072914 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:03.072914 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:03.072914 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:03.072914 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:03.072914 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:03.072914 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:03.072914 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:03.072914 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:03.072914 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:03.072914 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:03.072914 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:03.078506 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:03.078506 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:03.078506 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:03.078506 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:03.078506 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:03.078506 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:03.078506 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:03.078506 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:03.078506 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:03.078506 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:03.078506 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:28:03.078506 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:28:03.078506 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 34/73] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Aug 29 22:28:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:03.078506 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e31969-7200-4390-8664-eca94543c8df None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:03.079197 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e31969-7200-4390-8664-eca94543c8df None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:03.079197 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e31969-7200-4390-8664-eca94543c8df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:03.095506 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-c8e31969-7200-4390-8664-eca94543c8df None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:03.096884 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-c8e31969-7200-4390-8664-eca94543c8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:03.135823 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d17ce24-fda6-4203-b60f-409c3e1cab13 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:03.167247 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 35/74] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:03.168022 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e31969-7200-4390-8664-eca94543c8df None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:03.172438 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a3c95a-d77a-4629-83c4-b3d553ed4b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:03.173326 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a3c95a-d77a-4629-83c4-b3d553ed4b95 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:03.174350 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a3c95a-d77a-4629-83c4-b3d553ed4b95 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:03.175359 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a3c95a-d77a-4629-83c4-b3d553ed4b95 None admin] PATH_INFO: `/v3/users/3a00a8e520484973b5ffa1769dcee1fc` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:03.197208 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a3c95a-d77a-4629-83c4-b3d553ed4b95 None admin] RBAC: Authorizing `identity:get_user(user_id=3a00a8e520484973b5ffa1769dcee1fc)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:03.201539 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a3c95a-d77a-4629-83c4-b3d553ed4b95 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:03.206427 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 40/75] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:28:03 2022] GET /identity/v3/users/3a00a8e520484973b5ffa1769dcee1fc => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:28:03.206427 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a3c95a-d77a-4629-83c4-b3d553ed4b95 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:03.214856 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:03.245755 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:03.280397 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e31969-7200-4390-8664-eca94543c8df None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:03.317426 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36671377-bce6-4000-ba67-6ca0a21b32e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:03.318347 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36671377-bce6-4000-ba67-6ca0a21b32e0 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:03.318639 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36671377-bce6-4000-ba67-6ca0a21b32e0 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:03.318914 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36671377-bce6-4000-ba67-6ca0a21b32e0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:03.329912 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36671377-bce6-4000-ba67-6ca0a21b32e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:03.331092 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a3c95a-d77a-4629-83c4-b3d553ed4b95 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:03.335577 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-36671377-bce6-4000-ba67-6ca0a21b32e0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:28:03.335577 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:03.335577 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:03.335577 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:03.335577 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:03.335577 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:03.335577 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:03.335577 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:03.335577 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:03.335577 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:03.335577 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:03.335577 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:03.335577 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:03.335577 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:03.335577 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:03.335577 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:03.336395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:03.336395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:03.336395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:03.336395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:03.336395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:03.336395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:03.336395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:03.336395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:03.336395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:03.336395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:03.336395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:03.336395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:03.336395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:03.336395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:03.336395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:03.336395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:03.337209 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:03.337209 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:03.337209 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:03.337209 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:03.337209 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:03.337209 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:03.337209 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:03.337209 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:03.337209 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:03.337209 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:03.337209 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:28:03.337209 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:28:03.337209 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 36/76] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Aug 29 22:28:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:03.337861 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36671377-bce6-4000-ba67-6ca0a21b32e0 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:03.374817 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a157b94c-c1e3-463f-8e11-565085ce045a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:03.375427 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157b94c-c1e3-463f-8e11-565085ce045a None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:03.375707 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157b94c-c1e3-463f-8e11-565085ce045a None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:03.375982 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157b94c-c1e3-463f-8e11-565085ce045a None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:03.376650 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a157b94c-c1e3-463f-8e11-565085ce045a None admin] RBAC: Adding query filter params (name=admin) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:03.377049 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a157b94c-c1e3-463f-8e11-565085ce045a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:03.378431 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a157b94c-c1e3-463f-8e11-565085ce045a None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:03.388183 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 41/77] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:28:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:03.393224 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a157b94c-c1e3-463f-8e11-565085ce045a None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:03.458787 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36671377-bce6-4000-ba67-6ca0a21b32e0 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:03.460334 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:03.487269 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:03.493954 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411bf44c-7f7d-4703-96a7-98cdb4c6d241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:03.494584 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411bf44c-7f7d-4703-96a7-98cdb4c6d241 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:03.494866 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411bf44c-7f7d-4703-96a7-98cdb4c6d241 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:03.495140 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411bf44c-7f7d-4703-96a7-98cdb4c6d241 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:03.499295 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a157b94c-c1e3-463f-8e11-565085ce045a None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:03.505923 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411bf44c-7f7d-4703-96a7-98cdb4c6d241 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:03.510793 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-411bf44c-7f7d-4703-96a7-98cdb4c6d241 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:28:03.510793 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:03.510793 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:03.510793 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:03.510793 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:03.510793 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:03.510793 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:03.510793 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:03.510793 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:03.510793 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:03.510793 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:03.510793 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:03.510793 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:03.510793 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:03.510793 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:03.510793 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:03.511828 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:03.511828 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:03.511828 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:03.511828 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:03.511828 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:03.511828 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:03.511828 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:03.511828 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:03.511828 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:03.511828 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:03.511828 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:03.511828 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:03.511828 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:03.511828 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:03.511828 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:03.511828 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:03.512889 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:03.512889 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:03.512889 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:03.512889 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:03.512889 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:03.512889 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:03.512889 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:03.512889 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:03.512889 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:03.512889 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:03.512889 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:28:03.512889 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:28:03.512889 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 37/78] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:28:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:03.517422 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411bf44c-7f7d-4703-96a7-98cdb4c6d241 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:03.537381 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac33107-5ac8-4e54-b92b-0b30aa7c7f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:03.538012 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac33107-5ac8-4e54-b92b-0b30aa7c7f37 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:03.538304 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac33107-5ac8-4e54-b92b-0b30aa7c7f37 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:03.538886 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac33107-5ac8-4e54-b92b-0b30aa7c7f37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:03.607239 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac33107-5ac8-4e54-b92b-0b30aa7c7f37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:03.608712 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac33107-5ac8-4e54-b92b-0b30aa7c7f37 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:03.619997 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411bf44c-7f7d-4703-96a7-98cdb4c6d241 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:03.654717 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cbba6b-e7dc-4f4f-a602-d68cc399754d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:03.655319 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cbba6b-e7dc-4f4f-a602-d68cc399754d None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:03.655599 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cbba6b-e7dc-4f4f-a602-d68cc399754d None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:03.655866 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cbba6b-e7dc-4f4f-a602-d68cc399754d None admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:03.656526 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cbba6b-e7dc-4f4f-a602-d68cc399754d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:03.656858 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cbba6b-e7dc-4f4f-a602-d68cc399754d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:03.658850 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cbba6b-e7dc-4f4f-a602-d68cc399754d None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:03.666913 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 38/79] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:28:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:03.667716 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cbba6b-e7dc-4f4f-a602-d68cc399754d None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:03.693342 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:03.696952 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:03.714384 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 42/80] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:28:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:28:03.714384 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac33107-5ac8-4e54-b92b-0b30aa7c7f37 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:03.769336 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cbba6b-e7dc-4f4f-a602-d68cc399754d None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:03.804299 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3134af-cf52-4e16-aec1-f8540e68e017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:03.804957 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3134af-cf52-4e16-aec1-f8540e68e017 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:03.805241 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3134af-cf52-4e16-aec1-f8540e68e017 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:03.805523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3134af-cf52-4e16-aec1-f8540e68e017 None admin] PATH_INFO: `/v3/users/9cc82f35b5c947928c9caa5d0b7e11fc` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:03.806663 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3134af-cf52-4e16-aec1-f8540e68e017 None admin] RBAC: Authorizing `identity:get_user(user_id=9cc82f35b5c947928c9caa5d0b7e11fc)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:03.808106 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3134af-cf52-4e16-aec1-f8540e68e017 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:03.809736 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 39/81] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:28:03 2022] GET /identity/v3/users/9cc82f35b5c947928c9caa5d0b7e11fc => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:03.810502 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3134af-cf52-4e16-aec1-f8540e68e017 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:03.816695 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac33107-5ac8-4e54-b92b-0b30aa7c7f37 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:03.851415 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f1dfba-d5dc-4ca8-a0ac-b69f97456d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:03.852044 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1dfba-d5dc-4ca8-a0ac-b69f97456d6e None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:03.852332 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1dfba-d5dc-4ca8-a0ac-b69f97456d6e None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:03.852610 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1dfba-d5dc-4ca8-a0ac-b69f97456d6e None admin] PATH_INFO: `/v3/domains/default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:03.853754 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f1dfba-d5dc-4ca8-a0ac-b69f97456d6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:03.855190 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f1dfba-d5dc-4ca8-a0ac-b69f97456d6e None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:03.856784 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 43/82] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:28:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:03.857553 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f1dfba-d5dc-4ca8-a0ac-b69f97456d6e None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:03.906402 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:03.915688 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3134af-cf52-4e16-aec1-f8540e68e017 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:03.917858 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:03.950168 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de855b02-b7af-4034-8fe7-98f5181bee09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:03.950777 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de855b02-b7af-4034-8fe7-98f5181bee09 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:03.951059 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de855b02-b7af-4034-8fe7-98f5181bee09 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:03.951490 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de855b02-b7af-4034-8fe7-98f5181bee09 None admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:03.952170 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de855b02-b7af-4034-8fe7-98f5181bee09 None admin] RBAC: Adding query filter params (name=member) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:03.952479 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de855b02-b7af-4034-8fe7-98f5181bee09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:03.953897 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de855b02-b7af-4034-8fe7-98f5181bee09 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:03.963174 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f1dfba-d5dc-4ca8-a0ac-b69f97456d6e None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:03.963321 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 40/83] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Aug 29 22:28:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:03.964145 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de855b02-b7af-4034-8fe7-98f5181bee09 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:04.000063 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f723a44c-143b-420f-b7cd-1488a0cee874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:04.000623 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723a44c-143b-420f-b7cd-1488a0cee874 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:04.000937 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723a44c-143b-420f-b7cd-1488a0cee874 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:04.001216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723a44c-143b-420f-b7cd-1488a0cee874 None admin] PATH_INFO: `/v3/users/3a00a8e520484973b5ffa1769dcee1fc` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:04.002588 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f723a44c-143b-420f-b7cd-1488a0cee874 None admin] RBAC: Authorizing `identity:get_user(user_id=3a00a8e520484973b5ffa1769dcee1fc)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:04.004059 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f723a44c-143b-420f-b7cd-1488a0cee874 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:04.005685 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 44/84] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:28:03 2022] GET /identity/v3/users/3a00a8e520484973b5ffa1769dcee1fc => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:04.006457 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f723a44c-143b-420f-b7cd-1488a0cee874 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:04.068984 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de855b02-b7af-4034-8fe7-98f5181bee09 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:04.103864 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a2e80a-0bdd-47cb-8462-bb2eca353c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:04.104490 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2e80a-0bdd-47cb-8462-bb2eca353c98 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:04.104802 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2e80a-0bdd-47cb-8462-bb2eca353c98 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:04.105080 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2e80a-0bdd-47cb-8462-bb2eca353c98 None admin] PATH_INFO: `/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:04.106212 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a2e80a-0bdd-47cb-8462-bb2eca353c98 None admin] RBAC: Authorizing `identity:get_project(project_id=bd224ec63c3748aa90cf3e984e32fa3f)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:04.107649 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a2e80a-0bdd-47cb-8462-bb2eca353c98 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:04.108026 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f723a44c-143b-420f-b7cd-1488a0cee874 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:04.109305 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 41/85] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:28:03 2022] GET /identity/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:04.110036 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a2e80a-0bdd-47cb-8462-bb2eca353c98 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:04.137641 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:04.141628 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:04.154723 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e4a4e-b26a-4d2b-a91e-432f39f72991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:04.155341 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e4a4e-b26a-4d2b-a91e-432f39f72991 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:04.157168 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e4a4e-b26a-4d2b-a91e-432f39f72991 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:04.157168 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e4a4e-b26a-4d2b-a91e-432f39f72991 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:04.221054 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a2e80a-0bdd-47cb-8462-bb2eca353c98 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:04.225571 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e4a4e-b26a-4d2b-a91e-432f39f72991 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:04.227128 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e4a4e-b26a-4d2b-a91e-432f39f72991 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:04.255341 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d126bf-3512-4090-9430-4981ded69a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:04.255881 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d126bf-3512-4090-9430-4981ded69a82 None admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:04.256167 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d126bf-3512-4090-9430-4981ded69a82 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:04.256448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d126bf-3512-4090-9430-4981ded69a82 None admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:04.257153 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d126bf-3512-4090-9430-4981ded69a82 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:04.258608 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d126bf-3512-4090-9430-4981ded69a82 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:04.326356 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 45/86] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:28:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:28:04.327194 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e4a4e-b26a-4d2b-a91e-432f39f72991 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:04.350214 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 42/87] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Aug 29 22:28:04 2022] POST /identity/v3/users => generated 289 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:28:04.351049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d126bf-3512-4090-9430-4981ded69a82 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:04.351720 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:04.438890 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e4a4e-b26a-4d2b-a91e-432f39f72991 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:04.451197 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:04.475839 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d126bf-3512-4090-9430-4981ded69a82 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:04.481348 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2f98c-2e20-4ab2-875b-fd83422f89a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:04.482004 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2f98c-2e20-4ab2-875b-fd83422f89a2 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:04.482374 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2f98c-2e20-4ab2-875b-fd83422f89a2 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:04.482633 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2f98c-2e20-4ab2-875b-fd83422f89a2 None admin] PATH_INFO: `/v3/users/3a00a8e520484973b5ffa1769dcee1fc` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:04.483823 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff2f98c-2e20-4ab2-875b-fd83422f89a2 None admin] RBAC: Authorizing `identity:get_user(user_id=3a00a8e520484973b5ffa1769dcee1fc)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:04.485362 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff2f98c-2e20-4ab2-875b-fd83422f89a2 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:04.487639 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 46/88] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:28:04 2022] GET /identity/v3/users/3a00a8e520484973b5ffa1769dcee1fc => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:04.488436 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2f98c-2e20-4ab2-875b-fd83422f89a2 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:04.520502 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179009f5-26c3-4dda-bfbf-c0c5eccff0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:04.521105 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179009f5-26c3-4dda-bfbf-c0c5eccff0ff None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:04.521389 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179009f5-26c3-4dda-bfbf-c0c5eccff0ff None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:04.521668 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179009f5-26c3-4dda-bfbf-c0c5eccff0ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:04.606399 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:04.615597 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179009f5-26c3-4dda-bfbf-c0c5eccff0ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:04.617222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179009f5-26c3-4dda-bfbf-c0c5eccff0ff None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:04.619260 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2f98c-2e20-4ab2-875b-fd83422f89a2 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:04.665908 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b646178-aeff-43ec-8740-adff557d2828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:04.666523 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b646178-aeff-43ec-8740-adff557d2828 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:04.666805 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b646178-aeff-43ec-8740-adff557d2828 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:04.667077 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b646178-aeff-43ec-8740-adff557d2828 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:04.667854 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b646178-aeff-43ec-8740-adff557d2828 None admin] RBAC: Adding query filter params (role.id=ae06bb653e3b4990a9ddaa1a649c7e06, scope.project.id=bd224ec63c3748aa90cf3e984e32fa3f, user.id=3a00a8e520484973b5ffa1769dcee1fc) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:04.668168 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b646178-aeff-43ec-8740-adff557d2828 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:04.669669 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b646178-aeff-43ec-8740-adff557d2828 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:04.675843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 47/89] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:28:04 2022] GET /identity/v3/role_assignments?user.id=3a00a8e520484973b5ffa1769dcee1fc&scope.project.id=bd224ec63c3748aa90cf3e984e32fa3f&role.id=ae06bb653e3b4990a9ddaa1a649c7e06 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:04.676664 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b646178-aeff-43ec-8740-adff557d2828 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:04.718731 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:04.748933 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 43/90] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:28:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:28:04.749835 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179009f5-26c3-4dda-bfbf-c0c5eccff0ff None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:04.811697 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b646178-aeff-43ec-8740-adff557d2828 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:04.849135 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd513943-4945-41d3-9678-f6d1ab794957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:04.849779 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd513943-4945-41d3-9678-f6d1ab794957 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:04.850061 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd513943-4945-41d3-9678-f6d1ab794957 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:04.850338 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd513943-4945-41d3-9678-f6d1ab794957 None admin] PATH_INFO: `/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:04.851496 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd513943-4945-41d3-9678-f6d1ab794957 None admin] RBAC: Authorizing `identity:get_project(project_id=bd224ec63c3748aa90cf3e984e32fa3f)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:04.852978 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd513943-4945-41d3-9678-f6d1ab794957 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:04.854656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 48/91] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:28:04 2022] GET /identity/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:04.855452 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd513943-4945-41d3-9678-f6d1ab794957 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:04.872761 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179009f5-26c3-4dda-bfbf-c0c5eccff0ff None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:04.905719 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:04.925446 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42642a8a-daac-4718-8449-d08a380b440d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:04.926076 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42642a8a-daac-4718-8449-d08a380b440d None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:04.926358 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42642a8a-daac-4718-8449-d08a380b440d None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:04.926630 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42642a8a-daac-4718-8449-d08a380b440d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:04.980692 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:04.992780 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd513943-4945-41d3-9678-f6d1ab794957 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:05.006282 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42642a8a-daac-4718-8449-d08a380b440d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:05.007828 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42642a8a-daac-4718-8449-d08a380b440d None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:05.036094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfcc799-946e-45b8-8589-7b7342fe7e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:05.036849 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfcc799-946e-45b8-8589-7b7342fe7e2b None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:05.037135 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfcc799-946e-45b8-8589-7b7342fe7e2b None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:05.037413 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfcc799-946e-45b8-8589-7b7342fe7e2b None admin] PATH_INFO: `/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:05.038544 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfcc799-946e-45b8-8589-7b7342fe7e2b None admin] RBAC: Authorizing `identity:get_project(project_id=bd224ec63c3748aa90cf3e984e32fa3f)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:05.039991 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfcc799-946e-45b8-8589-7b7342fe7e2b None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:05.041772 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 49/92] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:28:04 2022] GET /identity/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:05.042539 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfcc799-946e-45b8-8589-7b7342fe7e2b None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:05.120734 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 44/93] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:28:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:28:05.121547 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42642a8a-daac-4718-8449-d08a380b440d None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:05.145211 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:05.154579 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfcc799-946e-45b8-8589-7b7342fe7e2b None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:05.190836 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d2bad-9a4a-4b20-bab9-92407bdc4000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:05.191448 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d2bad-9a4a-4b20-bab9-92407bdc4000 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:05.191730 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d2bad-9a4a-4b20-bab9-92407bdc4000 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:05.192004 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d2bad-9a4a-4b20-bab9-92407bdc4000 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:05.192739 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909d2bad-9a4a-4b20-bab9-92407bdc4000 None admin] RBAC: Adding query filter params (role.id=882f35bd847a406eb2222aeafc517a46, scope.project.id=bd224ec63c3748aa90cf3e984e32fa3f, user.id=9cc82f35b5c947928c9caa5d0b7e11fc) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:05.193055 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909d2bad-9a4a-4b20-bab9-92407bdc4000 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:05.194496 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909d2bad-9a4a-4b20-bab9-92407bdc4000 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:05.200229 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 50/94] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:28:05 2022] GET /identity/v3/role_assignments?user.id=9cc82f35b5c947928c9caa5d0b7e11fc&scope.project.id=bd224ec63c3748aa90cf3e984e32fa3f&role.id=882f35bd847a406eb2222aeafc517a46 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:05.201044 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d2bad-9a4a-4b20-bab9-92407bdc4000 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:05.262516 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:05.291671 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42642a8a-daac-4718-8449-d08a380b440d None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:05.312619 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d2bad-9a4a-4b20-bab9-92407bdc4000 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:05.350572 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda61a7-f536-44d7-abef-3fd9607a68a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:05.351404 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda61a7-f536-44d7-abef-3fd9607a68a7 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:05.351694 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda61a7-f536-44d7-abef-3fd9607a68a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:05.351970 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda61a7-f536-44d7-abef-3fd9607a68a7 None admin] PATH_INFO: `/v3/projects/62d4a24220ec4112bc3143d0a99fa0e3` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:05.353168 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda61a7-f536-44d7-abef-3fd9607a68a7 None admin] RBAC: Authorizing `identity:get_project(project_id=62d4a24220ec4112bc3143d0a99fa0e3)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:05.354689 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda61a7-f536-44d7-abef-3fd9607a68a7 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:05.356676 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 51/95] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:28:05 2022] GET /identity/v3/projects/62d4a24220ec4112bc3143d0a99fa0e3 => generated 312 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:05.361623 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf2792-2e14-4590-9fa0-862c37517e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:05.362277 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf2792-2e14-4590-9fa0-862c37517e7b None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:05.362511 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda61a7-f536-44d7-abef-3fd9607a68a7 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:05.362611 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf2792-2e14-4590-9fa0-862c37517e7b None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:05.362823 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf2792-2e14-4590-9fa0-862c37517e7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:05.363577 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caf2792-2e14-4590-9fa0-862c37517e7b None admin] RBAC: Adding query filter params (role.id=a6d1677ebe3a4355be5d67c5c78d83c7, scope.project.id=bd224ec63c3748aa90cf3e984e32fa3f, user.id=3a00a8e520484973b5ffa1769dcee1fc) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:05.363891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caf2792-2e14-4590-9fa0-862c37517e7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:05.365470 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caf2792-2e14-4590-9fa0-862c37517e7b None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:05.387119 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 45/96] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:28:05 2022] GET /identity/v3/role_assignments?user.id=3a00a8e520484973b5ffa1769dcee1fc&scope.project.id=bd224ec63c3748aa90cf3e984e32fa3f&role.id=a6d1677ebe3a4355be5d67c5c78d83c7 => generated 259 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:05.391179 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:05.481286 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda61a7-f536-44d7-abef-3fd9607a68a7 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:05.519359 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b8241a-a3cb-4aba-88e7-adfb6c213be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:05.519971 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8241a-a3cb-4aba-88e7-adfb6c213be3 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:05.520252 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8241a-a3cb-4aba-88e7-adfb6c213be3 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:05.520521 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8241a-a3cb-4aba-88e7-adfb6c213be3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:05.521279 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b8241a-a3cb-4aba-88e7-adfb6c213be3 None admin] RBAC: Adding query filter params (role.id=ae06bb653e3b4990a9ddaa1a649c7e06, scope.project.id=62d4a24220ec4112bc3143d0a99fa0e3, user.id=3a00a8e520484973b5ffa1769dcee1fc) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:05.521585 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b8241a-a3cb-4aba-88e7-adfb6c213be3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:05.523015 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b8241a-a3cb-4aba-88e7-adfb6c213be3 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:05.529084 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 52/97] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:28:05 2022] GET /identity/v3/role_assignments?user.id=3a00a8e520484973b5ffa1769dcee1fc&scope.project.id=62d4a24220ec4112bc3143d0a99fa0e3&role.id=ae06bb653e3b4990a9ddaa1a649c7e06 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:28:07.086820 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa3b3d-1268-4f3d-9d2d-af35d88b2cbf None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:07.087331 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa3b3d-1268-4f3d-9d2d-af35d88b2cbf None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:07.087433 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa3b3d-1268-4f3d-9d2d-af35d88b2cbf None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:07.088897 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 46/98] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:07.094249 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b67146-61f2-4beb-93b9-c390abfe1451 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:07.095034 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b67146-61f2-4beb-93b9-c390abfe1451 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:07.095511 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b67146-61f2-4beb-93b9-c390abfe1451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:07.116828 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-15b67146-61f2-4beb-93b9-c390abfe1451 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:07.118342 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-15b67146-61f2-4beb-93b9-c390abfe1451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:07.200498 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 53/99] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:07.237818 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87fba83-0c46-47fa-8ad0-ae15691f1d7a None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:07.238120 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87fba83-0c46-47fa-8ad0-ae15691f1d7a None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:07.238387 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87fba83-0c46-47fa-8ad0-ae15691f1d7a None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:07.239752 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 47/100] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:07.245181 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b354b-4a97-4815-8492-b5af16afa1da None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:07.245849 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b354b-4a97-4815-8492-b5af16afa1da None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:07.246322 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b354b-4a97-4815-8492-b5af16afa1da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:07.271115 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-612b354b-4a97-4815-8492-b5af16afa1da None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:07.272633 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-612b354b-4a97-4815-8492-b5af16afa1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:07.291998 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:07.356944 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 54/101] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 22:28:07.777560 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87fba83-0c46-47fa-8ad0-ae15691f1d7a None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:07.922671 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87fba83-0c46-47fa-8ad0-ae15691f1d7a None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:07.957714 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48129eb9-99c4-4285-b20e-2f2f86e4f81b None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:07.958123 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48129eb9-99c4-4285-b20e-2f2f86e4f81b None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:07.958265 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48129eb9-99c4-4285-b20e-2f2f86e4f81b None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:07.959810 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 55/102] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:28:07.969526 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dcd76f-5de2-4b72-928a-a4eff7833fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:07.970161 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dcd76f-5de2-4b72-928a-a4eff7833fe5 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:07.970443 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dcd76f-5de2-4b72-928a-a4eff7833fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:07.971050 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dcd76f-5de2-4b72-928a-a4eff7833fe5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:07.975997 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225a77d5-c48a-4d59-b93d-dc556892a3df None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:07.976287 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225a77d5-c48a-4d59-b93d-dc556892a3df None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:07.979048 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225a77d5-c48a-4d59-b93d-dc556892a3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:07.998826 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dcd76f-5de2-4b72-928a-a4eff7833fe5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:08.003176 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-f5dcd76f-5de2-4b72-928a-a4eff7833fe5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:28:08.003176 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:08.003176 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:08.003176 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:08.003176 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:08.003176 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:08.003176 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:08.003176 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:08.003176 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:08.003176 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:08.003176 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:08.003176 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:08.003176 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:08.003176 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:08.003176 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:08.003176 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:08.004614 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:08.004614 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:08.004614 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:08.004614 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:08.004614 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:08.004614 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:08.004614 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:08.004614 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:08.004614 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:08.004614 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:08.004614 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:08.004614 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:08.004614 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:08.004614 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:08.004614 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:08.004614 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:08.006089 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:08.006089 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:08.006089 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:08.006089 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:08.006089 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:08.006089 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:08.006089 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:08.006089 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:08.006089 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:08.006089 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:08.006089 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:28:08.006089 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:28:08.006089 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 48/103] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:28:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:28:08.013185 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dcd76f-5de2-4b72-928a-a4eff7833fe5 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:08.019378 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:08.025765 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-225a77d5-c48a-4d59-b93d-dc556892a3df None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:08.027001 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-225a77d5-c48a-4d59-b93d-dc556892a3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:08.133292 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 56/104] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 22:28:08.135078 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225a77d5-c48a-4d59-b93d-dc556892a3df None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:08.158638 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dcd76f-5de2-4b72-928a-a4eff7833fe5 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:08.217982 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f740219b-9fb3-40bb-887b-8953c0fb07f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:08.218605 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740219b-9fb3-40bb-887b-8953c0fb07f7 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:08.218888 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740219b-9fb3-40bb-887b-8953c0fb07f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:08.219166 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740219b-9fb3-40bb-887b-8953c0fb07f7 None admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:08.219849 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f740219b-9fb3-40bb-887b-8953c0fb07f7 None admin] RBAC: Adding query filter params (name=member) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:08.220155 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f740219b-9fb3-40bb-887b-8953c0fb07f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:08.222631 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f740219b-9fb3-40bb-887b-8953c0fb07f7 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:08.238876 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 49/105] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Aug 29 22:28:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:08.240135 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd61f8-d01c-467c-8947-2e48069bf638 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:08.240437 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd61f8-d01c-467c-8947-2e48069bf638 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:08.240743 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd61f8-d01c-467c-8947-2e48069bf638 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:08.242157 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 50/106] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:08.249812 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fd61f8-d01c-467c-8947-2e48069bf638 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:08.261176 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:08.285383 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225a77d5-c48a-4d59-b93d-dc556892a3df None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:08.337222 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf3c54a-7e1a-4874-b71a-c15f3eaea92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:08.338002 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:08.338344 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf3c54a-7e1a-4874-b71a-c15f3eaea92f None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:08.338631 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf3c54a-7e1a-4874-b71a-c15f3eaea92f None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:08.338908 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf3c54a-7e1a-4874-b71a-c15f3eaea92f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:08.362070 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf3c54a-7e1a-4874-b71a-c15f3eaea92f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:08.372388 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-9cf3c54a-7e1a-4874-b71a-c15f3eaea92f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 22:28:08.372388 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:08.372388 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:08.372388 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:08.372388 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:08.372388 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:08.372388 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:08.372388 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:08.372388 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:08.372388 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:08.372388 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:08.372388 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:08.372388 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:08.372388 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:08.372388 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:08.372388 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:08.373238 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:08.373238 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:08.373238 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:08.373238 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:08.373238 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:08.373238 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:08.373238 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:08.373238 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:08.373238 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:08.373238 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:08.373238 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:08.373238 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:08.373238 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:08.373238 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:08.373238 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:08.373238 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:08.374013 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:08.374013 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:08.374013 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:08.374013 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:08.374013 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:08.374013 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:08.374013 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:08.374013 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:08.374013 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:08.374013 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:08.374013 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 22:28:08.374013 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:28:08.374013 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 57/107] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:28:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:08.375381 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa4d42-ad00-408f-92f1-c6700f68e5cf None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:08.378799 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa4d42-ad00-408f-92f1-c6700f68e5cf None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:08.378799 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa4d42-ad00-408f-92f1-c6700f68e5cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:08.381196 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fd61f8-d01c-467c-8947-2e48069bf638 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:08.417827 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-d5aa4d42-ad00-408f-92f1-c6700f68e5cf None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:08.418790 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18b49f-76eb-46c6-8fb4-8e40801335d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:08.419026 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-d5aa4d42-ad00-408f-92f1-c6700f68e5cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:08.419561 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18b49f-76eb-46c6-8fb4-8e40801335d3 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:08.419916 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18b49f-76eb-46c6-8fb4-8e40801335d3 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:08.420253 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18b49f-76eb-46c6-8fb4-8e40801335d3 None admin] PATH_INFO: `/v3/users/3a00a8e520484973b5ffa1769dcee1fc` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:08.421417 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed18b49f-76eb-46c6-8fb4-8e40801335d3 None admin] RBAC: Authorizing `identity:get_user(user_id=3a00a8e520484973b5ffa1769dcee1fc)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:08.423142 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed18b49f-76eb-46c6-8fb4-8e40801335d3 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:08.425067 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 51/108] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:28:08 2022] GET /identity/v3/users/3a00a8e520484973b5ffa1769dcee1fc => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:08.425887 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18b49f-76eb-46c6-8fb4-8e40801335d3 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:08.525426 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 58/109] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:08.533809 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa4d42-ad00-408f-92f1-c6700f68e5cf None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:08.547764 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18b49f-76eb-46c6-8fb4-8e40801335d3 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:08.580105 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:08.589242 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da7172-5984-4b00-a0fb-f0b025a12cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:08.590051 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da7172-5984-4b00-a0fb-f0b025a12cc4 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:08.591453 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da7172-5984-4b00-a0fb-f0b025a12cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:08.591877 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da7172-5984-4b00-a0fb-f0b025a12cc4 None admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:08.592699 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61da7172-5984-4b00-a0fb-f0b025a12cc4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:08.593374 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61da7172-5984-4b00-a0fb-f0b025a12cc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:08.595011 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61da7172-5984-4b00-a0fb-f0b025a12cc4 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:08.615837 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 52/110] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Aug 29 22:28:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:08.618210 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da7172-5984-4b00-a0fb-f0b025a12cc4 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:08.647180 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:08.692283 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa4d42-ad00-408f-92f1-c6700f68e5cf None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:08.733383 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc283f6c-d62c-431f-ac41-054a08efeb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:08.734072 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc283f6c-d62c-431f-ac41-054a08efeb95 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:08.734361 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc283f6c-d62c-431f-ac41-054a08efeb95 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:08.734641 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc283f6c-d62c-431f-ac41-054a08efeb95 None admin] PATH_INFO: `/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:08.735780 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc283f6c-d62c-431f-ac41-054a08efeb95 None admin] RBAC: Authorizing `identity:get_project(project_id=bd224ec63c3748aa90cf3e984e32fa3f)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:08.738007 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc283f6c-d62c-431f-ac41-054a08efeb95 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:08.739663 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 59/111] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:28:08 2022] GET /identity/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:08.740577 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc283f6c-d62c-431f-ac41-054a08efeb95 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:08.750815 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da7172-5984-4b00-a0fb-f0b025a12cc4 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:08.785494 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7425c276-b68e-4035-944d-0e32a4627945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:08.786136 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7425c276-b68e-4035-944d-0e32a4627945 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:08.786423 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7425c276-b68e-4035-944d-0e32a4627945 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:08.786704 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7425c276-b68e-4035-944d-0e32a4627945 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:08.787821 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7425c276-b68e-4035-944d-0e32a4627945 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:08.789336 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7425c276-b68e-4035-944d-0e32a4627945 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:08.790907 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 53/112] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:28:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:08.792018 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8366a-c81c-4e03-8a40-ef24ff272ed4 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:08.792308 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8366a-c81c-4e03-8a40-ef24ff272ed4 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:08.792587 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8366a-c81c-4e03-8a40-ef24ff272ed4 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:08.793825 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 54/113] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:08.794507 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a8366a-c81c-4e03-8a40-ef24ff272ed4 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:08.862604 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc283f6c-d62c-431f-ac41-054a08efeb95 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:08.868004 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:08.901264 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ced521-05d0-450c-bff2-26b779006c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:08.901899 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ced521-05d0-450c-bff2-26b779006c4c None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:08.902188 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ced521-05d0-450c-bff2-26b779006c4c None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:08.902467 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ced521-05d0-450c-bff2-26b779006c4c None admin] PATH_INFO: `/v3/users/e3d531be164e4c969fd482fbc6c8c7f5` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:08.905592 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:08.911997 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a8366a-c81c-4e03-8a40-ef24ff272ed4 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:08.916994 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ced521-05d0-450c-bff2-26b779006c4c None admin] RBAC: Authorizing `identity:get_user(user_id=e3d531be164e4c969fd482fbc6c8c7f5)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:08.918473 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ced521-05d0-450c-bff2-26b779006c4c None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:08.920115 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 60/114] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:28:08 2022] GET /identity/v3/users/e3d531be164e4c969fd482fbc6c8c7f5 => generated 289 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:08.921311 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14fbc23-4927-4265-b7f7-31e836b79e0f None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:08.922101 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14fbc23-4927-4265-b7f7-31e836b79e0f None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:08.922101 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14fbc23-4927-4265-b7f7-31e836b79e0f None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:08.923318 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 61/115] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:08.924194 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14fbc23-4927-4265-b7f7-31e836b79e0f None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:08.954054 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22915927-89e7-4f81-bd00-5e05da5ea39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:08.954670 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22915927-89e7-4f81-bd00-5e05da5ea39f None admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:08.955923 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22915927-89e7-4f81-bd00-5e05da5ea39f None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:08.956066 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22915927-89e7-4f81-bd00-5e05da5ea39f None admin] PATH_INFO: `/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f/users/3a00a8e520484973b5ffa1769dcee1fc/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:08.958013 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22915927-89e7-4f81-bd00-5e05da5ea39f None admin] RBAC: Authorizing `identity:create_grant(project_id=bd224ec63c3748aa90cf3e984e32fa3f, user_id=3a00a8e520484973b5ffa1769dcee1fc, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:08.959474 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22915927-89e7-4f81-bd00-5e05da5ea39f None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:09.024742 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14fbc23-4927-4265-b7f7-31e836b79e0f None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:09.059095 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a668ae-3c53-49c4-8d94-a1b08aa0b26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:09.059697 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a668ae-3c53-49c4-8d94-a1b08aa0b26f None admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:09.059978 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a668ae-3c53-49c4-8d94-a1b08aa0b26f None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:09.060253 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a668ae-3c53-49c4-8d94-a1b08aa0b26f None admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:09.060929 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a668ae-3c53-49c4-8d94-a1b08aa0b26f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:09.062358 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a668ae-3c53-49c4-8d94-a1b08aa0b26f None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:09.075932 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 55/116] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Aug 29 22:28:08 2022] PUT /identity/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f/users/3a00a8e520484973b5ffa1769dcee1fc/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:28:09.077255 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb394ad-0008-400a-affd-cdcad3242fd7 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:09.077525 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb394ad-0008-400a-affd-cdcad3242fd7 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:09.077822 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb394ad-0008-400a-affd-cdcad3242fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:09.100146 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-bbb394ad-0008-400a-affd-cdcad3242fd7 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:09.101436 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-bbb394ad-0008-400a-affd-cdcad3242fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:09.181644 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 62/117] 10.222.0.52 () {60 vars in 1141 bytes} [Mon Aug 29 22:28:08 2022] POST /identity/v3/projects => generated 302 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:28:09.189038 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 56/118] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:09.189656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a668ae-3c53-49c4-8d94-a1b08aa0b26f None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:09.189735 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb394ad-0008-400a-affd-cdcad3242fd7 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:09.277889 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:09.292599 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:09.359299 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a668ae-3c53-49c4-8d94-a1b08aa0b26f None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:09.365528 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb394ad-0008-400a-affd-cdcad3242fd7 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:09.410638 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6002ebf-746b-4c9b-9683-0743dbedb99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:09.411257 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6002ebf-746b-4c9b-9683-0743dbedb99e None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:09.411546 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6002ebf-746b-4c9b-9683-0743dbedb99e None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:09.411825 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6002ebf-746b-4c9b-9683-0743dbedb99e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:09.412994 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30012fe4-9717-4393-b118-839b62cc4c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:09.413630 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30012fe4-9717-4393-b118-839b62cc4c4a None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:09.413910 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30012fe4-9717-4393-b118-839b62cc4c4a None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:09.414185 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30012fe4-9717-4393-b118-839b62cc4c4a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:09.428497 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30012fe4-9717-4393-b118-839b62cc4c4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:09.435721 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-30012fe4-9717-4393-b118-839b62cc4c4a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:28:09.435721 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:09.435721 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:09.435721 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:09.435721 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:09.435721 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:09.435721 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:09.435721 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:09.435721 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:09.435721 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:09.435721 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:09.435721 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:09.435721 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:09.435721 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:09.435721 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:09.435721 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:09.436550 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:09.436550 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:09.436550 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:09.436550 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:09.436550 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:09.436550 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:09.436550 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:09.436550 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:09.436550 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:09.436550 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:09.436550 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:09.436550 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:09.436550 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:09.436550 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:09.436550 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:09.436550 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:09.437953 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:09.437953 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:09.437953 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:09.437953 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:09.437953 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:09.437953 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:09.437953 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:09.437953 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:09.437953 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:09.437953 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:09.437953 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:28:09.437953 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:28:09.437953 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 63/119] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Aug 29 22:28:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:09.441829 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f0728-075e-4b2b-a630-2edb03d74d57 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:09.441829 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f0728-075e-4b2b-a630-2edb03d74d57 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:09.441829 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f0728-075e-4b2b-a630-2edb03d74d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:09.463810 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-847f0728-075e-4b2b-a630-2edb03d74d57 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:09.465666 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-847f0728-075e-4b2b-a630-2edb03d74d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:09.494519 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6002ebf-746b-4c9b-9683-0743dbedb99e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:09.496242 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6002ebf-746b-4c9b-9683-0743dbedb99e None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:09.568925 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 64/120] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:09.569923 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f0728-075e-4b2b-a630-2edb03d74d57 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:09.580760 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:09.650227 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:09.693310 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 57/121] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:28:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:28:09.699976 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6002ebf-746b-4c9b-9683-0743dbedb99e None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:09.781420 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f0728-075e-4b2b-a630-2edb03d74d57 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:09.815262 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6002ebf-746b-4c9b-9683-0743dbedb99e None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:09.858649 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bb2539-8050-4df9-9d91-b946050dc316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:09.858649 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb2539-8050-4df9-9d91-b946050dc316 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:09.858649 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb2539-8050-4df9-9d91-b946050dc316 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:09.858982 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb2539-8050-4df9-9d91-b946050dc316 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:09.871108 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3472895c-2b25-4e68-8450-1aca215dc5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:09.872024 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472895c-2b25-4e68-8450-1aca215dc5a1 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:09.872486 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472895c-2b25-4e68-8450-1aca215dc5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:09.873162 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472895c-2b25-4e68-8450-1aca215dc5a1 None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:09.874001 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3472895c-2b25-4e68-8450-1aca215dc5a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:09.875484 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3472895c-2b25-4e68-8450-1aca215dc5a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:09.877876 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3472895c-2b25-4e68-8450-1aca215dc5a1 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:09.883653 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bb2539-8050-4df9-9d91-b946050dc316 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:09.888910 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-97bb2539-8050-4df9-9d91-b946050dc316 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:28:09.888910 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:09.888910 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:09.888910 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:09.888910 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:09.888910 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:09.888910 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:09.888910 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:09.888910 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:09.888910 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:09.888910 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:09.888910 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:09.888910 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:09.888910 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:09.888910 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:09.888910 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:09.889724 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:09.889724 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:09.889724 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:09.889724 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:09.889724 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:09.889724 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:09.889724 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:09.889724 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:09.889724 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:09.889724 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:09.889724 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:09.889724 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:09.889724 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:09.889724 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:09.889724 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:09.889724 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:09.890478 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:09.890478 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:09.890478 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:09.890478 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:09.890478 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:09.890478 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:09.890478 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:09.890478 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:09.890478 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:09.890478 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:09.890478 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:28:09.890478 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:28:09.901142 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 58/122] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Aug 29 22:28:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:09.902343 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bb2539-8050-4df9-9d91-b946050dc316 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:09.904530 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 65/123] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:28:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:09.906981 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3472895c-2b25-4e68-8450-1aca215dc5a1 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:09.929139 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:10.024549 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bb2539-8050-4df9-9d91-b946050dc316 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:10.034579 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:10.040943 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3472895c-2b25-4e68-8450-1aca215dc5a1 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:10.067340 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962564fc-62b2-4799-af39-eb2a0e70de91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:10.068076 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962564fc-62b2-4799-af39-eb2a0e70de91 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:10.068346 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962564fc-62b2-4799-af39-eb2a0e70de91 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:10.068688 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962564fc-62b2-4799-af39-eb2a0e70de91 None admin] PATH_INFO: `/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:10.070205 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962564fc-62b2-4799-af39-eb2a0e70de91 None admin] RBAC: Authorizing `identity:get_project(project_id=bd224ec63c3748aa90cf3e984e32fa3f)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:10.072223 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962564fc-62b2-4799-af39-eb2a0e70de91 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:10.074273 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 59/124] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:28:09 2022] GET /identity/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:10.079048 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962564fc-62b2-4799-af39-eb2a0e70de91 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:10.089153 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb9875b-b285-4973-9c10-59a5e34c8c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:10.090112 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9875b-b285-4973-9c10-59a5e34c8c10 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:10.090588 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9875b-b285-4973-9c10-59a5e34c8c10 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:10.091075 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9875b-b285-4973-9c10-59a5e34c8c10 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:10.104095 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb9875b-b285-4973-9c10-59a5e34c8c10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:10.109976 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-2eb9875b-b285-4973-9c10-59a5e34c8c10 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:28:10.109976 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:10.109976 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:10.109976 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:10.109976 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:10.109976 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:10.109976 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:10.109976 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:10.109976 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:10.109976 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:10.109976 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:10.109976 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:10.109976 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:10.109976 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:10.109976 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:10.109976 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:10.110820 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:10.110820 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:10.110820 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:10.110820 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:10.110820 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:10.110820 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:10.110820 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:10.110820 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:10.110820 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:10.110820 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:10.110820 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:10.110820 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:10.110820 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:10.110820 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:10.110820 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:10.110820 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:10.112449 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:10.112449 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:10.112449 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:10.112449 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:10.112449 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:10.112449 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:10.112449 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:10.112449 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:10.112449 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:10.112449 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:10.112449 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:28:10.112449 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:28:10.114813 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 66/125] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:28:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:10.115977 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb9875b-b285-4973-9c10-59a5e34c8c10 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:10.184120 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:10.215795 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962564fc-62b2-4799-af39-eb2a0e70de91 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:10.242864 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb9875b-b285-4973-9c10-59a5e34c8c10 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:10.256244 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f6400d-b4a1-435a-8afb-523a18ad2807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:10.256923 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6400d-b4a1-435a-8afb-523a18ad2807 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:10.257292 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6400d-b4a1-435a-8afb-523a18ad2807 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:10.257562 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6400d-b4a1-435a-8afb-523a18ad2807 None admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:10.258242 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f6400d-b4a1-435a-8afb-523a18ad2807 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:10.258550 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f6400d-b4a1-435a-8afb-523a18ad2807 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:10.260182 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f6400d-b4a1-435a-8afb-523a18ad2807 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:10.274371 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 60/126] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:28:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:10.274371 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f6400d-b4a1-435a-8afb-523a18ad2807 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:10.288169 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9c366-2584-45cc-a528-c621f6538234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:10.289269 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9c366-2584-45cc-a528-c621f6538234 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:10.289749 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9c366-2584-45cc-a528-c621f6538234 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:10.290216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9c366-2584-45cc-a528-c621f6538234 None admin] PATH_INFO: `/v3/users/9cc82f35b5c947928c9caa5d0b7e11fc` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:10.291599 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d9c366-2584-45cc-a528-c621f6538234 None admin] RBAC: Authorizing `identity:get_user(user_id=9cc82f35b5c947928c9caa5d0b7e11fc)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:10.293323 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d9c366-2584-45cc-a528-c621f6538234 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:10.295091 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 67/127] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:28:10 2022] GET /identity/v3/users/9cc82f35b5c947928c9caa5d0b7e11fc => generated 247 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:10.296137 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9c366-2584-45cc-a528-c621f6538234 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:10.327685 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:10.387697 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f6400d-b4a1-435a-8afb-523a18ad2807 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:10.409490 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9c366-2584-45cc-a528-c621f6538234 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:10.428054 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf6e4b-ebfb-4c80-b72c-60f9e22f7305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:10.428672 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf6e4b-ebfb-4c80-b72c-60f9e22f7305 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:10.428992 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf6e4b-ebfb-4c80-b72c-60f9e22f7305 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:10.429518 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf6e4b-ebfb-4c80-b72c-60f9e22f7305 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:10.430082 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf6e4b-ebfb-4c80-b72c-60f9e22f7305 None admin] RBAC: Adding query filter params (role.id=b9691ffb7caa429ab9a5876dd3dc2ba3, scope.project.id=bd224ec63c3748aa90cf3e984e32fa3f, user.id=e3d531be164e4c969fd482fbc6c8c7f5) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:10.430393 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf6e4b-ebfb-4c80-b72c-60f9e22f7305 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:10.431870 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf6e4b-ebfb-4c80-b72c-60f9e22f7305 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:10.444747 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 61/128] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:28:10 2022] GET /identity/v3/role_assignments?user.id=e3d531be164e4c969fd482fbc6c8c7f5&scope.project.id=bd224ec63c3748aa90cf3e984e32fa3f&role.id=b9691ffb7caa429ab9a5876dd3dc2ba3 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:10.444747 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf6e4b-ebfb-4c80-b72c-60f9e22f7305 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:10.448836 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:10.460373 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c556e28a-1a7a-4afd-bcae-ac850a852e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:10.461150 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c556e28a-1a7a-4afd-bcae-ac850a852e9c None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:10.462979 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c556e28a-1a7a-4afd-bcae-ac850a852e9c None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:10.462979 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c556e28a-1a7a-4afd-bcae-ac850a852e9c None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:10.462979 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c556e28a-1a7a-4afd-bcae-ac850a852e9c None admin] RBAC: Adding query filter params (name=member) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:10.463386 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c556e28a-1a7a-4afd-bcae-ac850a852e9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:10.465258 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c556e28a-1a7a-4afd-bcae-ac850a852e9c None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:10.481011 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 68/129] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Aug 29 22:28:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:10.486218 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c556e28a-1a7a-4afd-bcae-ac850a852e9c None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:10.568957 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf6e4b-ebfb-4c80-b72c-60f9e22f7305 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:10.594457 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:10.612750 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c556e28a-1a7a-4afd-bcae-ac850a852e9c None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:10.625885 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da943b9-bc8d-44ff-8dab-6a30ac20df17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:10.626714 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da943b9-bc8d-44ff-8dab-6a30ac20df17 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:10.627714 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da943b9-bc8d-44ff-8dab-6a30ac20df17 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:10.628131 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da943b9-bc8d-44ff-8dab-6a30ac20df17 None admin] PATH_INFO: `/v3/users/3a00a8e520484973b5ffa1769dcee1fc` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:10.629740 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da943b9-bc8d-44ff-8dab-6a30ac20df17 None admin] RBAC: Authorizing `identity:get_user(user_id=3a00a8e520484973b5ffa1769dcee1fc)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:10.632365 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da943b9-bc8d-44ff-8dab-6a30ac20df17 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:10.634852 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 62/130] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:28:10 2022] GET /identity/v3/users/3a00a8e520484973b5ffa1769dcee1fc => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:10.635833 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da943b9-bc8d-44ff-8dab-6a30ac20df17 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:10.652833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4550ebc6-80eb-4959-9ab8-d48f914b38f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:10.653587 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4550ebc6-80eb-4959-9ab8-d48f914b38f7 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:10.653966 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4550ebc6-80eb-4959-9ab8-d48f914b38f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:10.654334 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4550ebc6-80eb-4959-9ab8-d48f914b38f7 None admin] PATH_INFO: `/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:10.656184 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4550ebc6-80eb-4959-9ab8-d48f914b38f7 None admin] RBAC: Authorizing `identity:get_project(project_id=bd224ec63c3748aa90cf3e984e32fa3f)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:10.657911 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4550ebc6-80eb-4959-9ab8-d48f914b38f7 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:10.661110 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 69/131] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:28:10 2022] GET /identity/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:10.662116 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4550ebc6-80eb-4959-9ab8-d48f914b38f7 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:10.724481 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:10.753100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da943b9-bc8d-44ff-8dab-6a30ac20df17 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:10.785710 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4550ebc6-80eb-4959-9ab8-d48f914b38f7 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:10.790519 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531baccb-e507-432c-86c4-b4d7e0b9a7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:10.791407 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531baccb-e507-432c-86c4-b4d7e0b9a7be None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:10.791881 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531baccb-e507-432c-86c4-b4d7e0b9a7be None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:10.792372 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531baccb-e507-432c-86c4-b4d7e0b9a7be None admin] PATH_INFO: `/v3/users/3a00a8e520484973b5ffa1769dcee1fc` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:10.793833 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531baccb-e507-432c-86c4-b4d7e0b9a7be None admin] RBAC: Authorizing `identity:get_user(user_id=3a00a8e520484973b5ffa1769dcee1fc)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:10.795648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531baccb-e507-432c-86c4-b4d7e0b9a7be None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:10.797813 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 63/132] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:28:10 2022] GET /identity/v3/users/3a00a8e520484973b5ffa1769dcee1fc => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:10.798772 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531baccb-e507-432c-86c4-b4d7e0b9a7be None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:10.822826 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fa9af5-418a-49ca-a6f2-09e0b5225024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:10.823361 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa9af5-418a-49ca-a6f2-09e0b5225024 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:10.823646 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa9af5-418a-49ca-a6f2-09e0b5225024 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:10.823926 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa9af5-418a-49ca-a6f2-09e0b5225024 None admin] PATH_INFO: `/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:10.825163 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fa9af5-418a-49ca-a6f2-09e0b5225024 None admin] RBAC: Authorizing `identity:get_project(project_id=bd224ec63c3748aa90cf3e984e32fa3f)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:10.826660 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fa9af5-418a-49ca-a6f2-09e0b5225024 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:10.828394 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 70/133] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:28:10 2022] GET /identity/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:10.833661 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fa9af5-418a-49ca-a6f2-09e0b5225024 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:10.864533 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:10.912386 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531baccb-e507-432c-86c4-b4d7e0b9a7be None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:10.944619 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fa9af5-418a-49ca-a6f2-09e0b5225024 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:10.949249 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1744e8a9-cd94-4ab4-a49c-11643f21d25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:10.950148 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744e8a9-cd94-4ab4-a49c-11643f21d25f None admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:10.950619 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744e8a9-cd94-4ab4-a49c-11643f21d25f None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:10.951080 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744e8a9-cd94-4ab4-a49c-11643f21d25f None admin] PATH_INFO: `/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f/users/9cc82f35b5c947928c9caa5d0b7e11fc/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:10.953382 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1744e8a9-cd94-4ab4-a49c-11643f21d25f None admin] RBAC: Authorizing `identity:create_grant(project_id=bd224ec63c3748aa90cf3e984e32fa3f, user_id=9cc82f35b5c947928c9caa5d0b7e11fc, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:10.955896 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1744e8a9-cd94-4ab4-a49c-11643f21d25f None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:10.983062 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562be9a4-e85e-4c12-ab2c-a02a831581eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:10.983953 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562be9a4-e85e-4c12-ab2c-a02a831581eb None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:10.984294 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562be9a4-e85e-4c12-ab2c-a02a831581eb None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:10.984635 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562be9a4-e85e-4c12-ab2c-a02a831581eb None admin] PATH_INFO: `/v3/projects/62d4a24220ec4112bc3143d0a99fa0e3` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:10.986048 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562be9a4-e85e-4c12-ab2c-a02a831581eb None admin] RBAC: Authorizing `identity:get_project(project_id=62d4a24220ec4112bc3143d0a99fa0e3)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:10.987939 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562be9a4-e85e-4c12-ab2c-a02a831581eb None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:10.990398 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 71/134] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:28:10 2022] GET /identity/v3/projects/62d4a24220ec4112bc3143d0a99fa0e3 => generated 312 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:10.991260 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562be9a4-e85e-4c12-ab2c-a02a831581eb None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:11.047360 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 64/135] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Aug 29 22:28:10 2022] PUT /identity/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f/users/9cc82f35b5c947928c9caa5d0b7e11fc/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:28:11.048546 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1744e8a9-cd94-4ab4-a49c-11643f21d25f None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:11.055723 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:11.101803 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:11.111718 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562be9a4-e85e-4c12-ab2c-a02a831581eb None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:11.162924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f7e55-d85f-40e2-8dd3-e41eb590729f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:11.168312 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f7e55-d85f-40e2-8dd3-e41eb590729f None admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:11.168664 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f7e55-d85f-40e2-8dd3-e41eb590729f None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:11.168972 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f7e55-d85f-40e2-8dd3-e41eb590729f None admin] PATH_INFO: `/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f/users/3a00a8e520484973b5ffa1769dcee1fc/roles/a6d1677ebe3a4355be5d67c5c78d83c7` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:11.176833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75f7e55-d85f-40e2-8dd3-e41eb590729f None admin] RBAC: Authorizing `identity:create_grant(project_id=bd224ec63c3748aa90cf3e984e32fa3f, user_id=3a00a8e520484973b5ffa1769dcee1fc, role_id=a6d1677ebe3a4355be5d67c5c78d83c7)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:11.180261 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75f7e55-d85f-40e2-8dd3-e41eb590729f None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:11.247072 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1744e8a9-cd94-4ab4-a49c-11643f21d25f None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:11.266258 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 72/136] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Aug 29 22:28:10 2022] PUT /identity/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f/users/3a00a8e520484973b5ffa1769dcee1fc/roles/a6d1677ebe3a4355be5d67c5c78d83c7 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:28:11.306780 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07fb1e-e70b-40a8-bcfe-c3bef6d6f741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:11.307920 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07fb1e-e70b-40a8-bcfe-c3bef6d6f741 None admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:11.308380 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07fb1e-e70b-40a8-bcfe-c3bef6d6f741 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:11.308929 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07fb1e-e70b-40a8-bcfe-c3bef6d6f741 None admin] PATH_INFO: `/v3/projects/62d4a24220ec4112bc3143d0a99fa0e3/users/3a00a8e520484973b5ffa1769dcee1fc/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:11.312326 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e07fb1e-e70b-40a8-bcfe-c3bef6d6f741 None admin] RBAC: Authorizing `identity:create_grant(project_id=62d4a24220ec4112bc3143d0a99fa0e3, user_id=3a00a8e520484973b5ffa1769dcee1fc, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:11.315715 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e07fb1e-e70b-40a8-bcfe-c3bef6d6f741 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:11.365548 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522de0e3-b2aa-4d99-a6d1-f9cafe54cbb9 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:11.365905 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522de0e3-b2aa-4d99-a6d1-f9cafe54cbb9 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:11.366490 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522de0e3-b2aa-4d99-a6d1-f9cafe54cbb9 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:11.368194 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 73/137] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:28:11.373841 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2014dc-57ff-4053-80d2-f790e1889658 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:11.374129 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2014dc-57ff-4053-80d2-f790e1889658 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:11.374460 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2014dc-57ff-4053-80d2-f790e1889658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:11.388194 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 65/138] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Aug 29 22:28:11 2022] PUT /identity/v3/projects/62d4a24220ec4112bc3143d0a99fa0e3/users/3a00a8e520484973b5ffa1769dcee1fc/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:28:11.401207 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-bb2014dc-57ff-4053-80d2-f790e1889658 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:11.403398 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-bb2014dc-57ff-4053-80d2-f790e1889658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:11.503954 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 74/139] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 22:28:11.931720 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49b775-7183-44a2-9d7d-42a67d18da95 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:11.932061 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49b775-7183-44a2-9d7d-42a67d18da95 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:11.932235 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49b775-7183-44a2-9d7d-42a67d18da95 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:11.933749 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 66/140] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:11.938794 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1f692-3b7a-4913-a0fd-b5696f2fb484 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:11.939079 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1f692-3b7a-4913-a0fd-b5696f2fb484 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:11.939383 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1f692-3b7a-4913-a0fd-b5696f2fb484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:11.943154 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49b775-7183-44a2-9d7d-42a67d18da95 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:11.973881 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-aaa1f692-3b7a-4913-a0fd-b5696f2fb484 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:11.975069 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-aaa1f692-3b7a-4913-a0fd-b5696f2fb484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:12.051277 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:12.091360 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49b775-7183-44a2-9d7d-42a67d18da95 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:12.126835 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 75/141] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:12.138365 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035306ac-0c06-42e9-a5b1-da115983f5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:12.139015 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035306ac-0c06-42e9-a5b1-da115983f5ba None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:12.139297 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035306ac-0c06-42e9-a5b1-da115983f5ba None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:12.139571 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035306ac-0c06-42e9-a5b1-da115983f5ba None admin] PATH_INFO: `/v3/roles/member` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:12.161630 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035306ac-0c06-42e9-a5b1-da115983f5ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:12.167530 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-035306ac-0c06-42e9-a5b1-da115983f5ba None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:28:12.167530 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:12.167530 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:12.167530 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:12.167530 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:12.167530 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:12.167530 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:12.167530 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:12.167530 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:12.167530 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:12.167530 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:12.167530 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:12.167530 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:12.167530 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:12.167530 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:12.167530 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:12.168410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:12.168410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:12.168410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:12.168410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:12.168410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:12.168410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:12.168410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:12.168410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:12.168410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:12.168410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:12.168410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:12.168410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:12.168410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:12.168410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:12.168410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:12.168410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:12.169172 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:12.169172 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:12.169172 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:12.169172 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:12.169172 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:12.169172 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:12.169172 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:12.169172 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:12.169172 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:12.169172 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:12.169172 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:28:12.169172 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:28:12.169172 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 67/142] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:28:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:12.175751 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1f692-3b7a-4913-a0fd-b5696f2fb484 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:12.196237 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:12.311631 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1f692-3b7a-4913-a0fd-b5696f2fb484 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:12.354786 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd0f4af-9df3-4d0a-994d-f589c02bfdc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:12.355400 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd0f4af-9df3-4d0a-994d-f589c02bfdc8 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:12.355679 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd0f4af-9df3-4d0a-994d-f589c02bfdc8 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:12.355946 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd0f4af-9df3-4d0a-994d-f589c02bfdc8 None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:12.356615 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd0f4af-9df3-4d0a-994d-f589c02bfdc8 None admin] RBAC: Adding query filter params (name=member) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:12.356940 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd0f4af-9df3-4d0a-994d-f589c02bfdc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:12.358329 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd0f4af-9df3-4d0a-994d-f589c02bfdc8 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:12.372617 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 76/143] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Aug 29 22:28:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:28:12.373678 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035306ac-0c06-42e9-a5b1-da115983f5ba None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:12.517980 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035306ac-0c06-42e9-a5b1-da115983f5ba None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:12.545469 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:12.558609 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d2ae1a-f337-4473-91f5-d6329ac40a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:12.559238 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2ae1a-f337-4473-91f5-d6329ac40a87 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:12.559517 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2ae1a-f337-4473-91f5-d6329ac40a87 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:12.559792 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2ae1a-f337-4473-91f5-d6329ac40a87 None admin] PATH_INFO: `/v3/users/3a00a8e520484973b5ffa1769dcee1fc` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:12.562278 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d2ae1a-f337-4473-91f5-d6329ac40a87 None admin] RBAC: Authorizing `identity:get_user(user_id=3a00a8e520484973b5ffa1769dcee1fc)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:12.562674 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d2ae1a-f337-4473-91f5-d6329ac40a87 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:12.564562 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 68/144] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:28:12 2022] GET /identity/v3/users/3a00a8e520484973b5ffa1769dcee1fc => generated 275 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:12.570311 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd0f4af-9df3-4d0a-994d-f589c02bfdc8 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:12.657952 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:12.670948 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d2ae1a-f337-4473-91f5-d6329ac40a87 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:12.698793 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd0f4af-9df3-4d0a-994d-f589c02bfdc8 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:12.735434 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc9ab4-8603-4a08-acfe-6bf9d58e6343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:12.736054 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc9ab4-8603-4a08-acfe-6bf9d58e6343 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:12.736369 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc9ab4-8603-4a08-acfe-6bf9d58e6343 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:12.736648 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc9ab4-8603-4a08-acfe-6bf9d58e6343 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:12.812479 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adc9ab4-8603-4a08-acfe-6bf9d58e6343 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:12.814078 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adc9ab4-8603-4a08-acfe-6bf9d58e6343 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:12.871798 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d2ae1a-f337-4473-91f5-d6329ac40a87 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:12.921087 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:12.948353 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a49ae-8832-47f1-8312-6aa2a84a5046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:12.949052 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a49ae-8832-47f1-8312-6aa2a84a5046 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:12.949336 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a49ae-8832-47f1-8312-6aa2a84a5046 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:12.949617 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a49ae-8832-47f1-8312-6aa2a84a5046 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:12.952026 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782a49ae-8832-47f1-8312-6aa2a84a5046 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:12.953549 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782a49ae-8832-47f1-8312-6aa2a84a5046 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:12.960142 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 69/145] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:28:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:12.961725 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a49ae-8832-47f1-8312-6aa2a84a5046 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:12.981151 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 77/146] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:28:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:28:12.988663 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc9ab4-8603-4a08-acfe-6bf9d58e6343 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:13.069510 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:13.103276 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc9ab4-8603-4a08-acfe-6bf9d58e6343 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:13.145287 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a49ae-8832-47f1-8312-6aa2a84a5046 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:13.155445 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cb9ed7-4e3f-4e7b-af7b-1364d0f14833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:13.156328 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb9ed7-4e3f-4e7b-af7b-1364d0f14833 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:13.156783 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb9ed7-4e3f-4e7b-af7b-1364d0f14833 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:13.157208 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb9ed7-4e3f-4e7b-af7b-1364d0f14833 None admin] PATH_INFO: `/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:13.161450 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cb9ed7-4e3f-4e7b-af7b-1364d0f14833 None admin] RBAC: Authorizing `identity:get_project(project_id=bd224ec63c3748aa90cf3e984e32fa3f)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:13.163245 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cb9ed7-4e3f-4e7b-af7b-1364d0f14833 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:13.168832 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 78/147] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:28:12 2022] GET /identity/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:13.172978 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cb9ed7-4e3f-4e7b-af7b-1364d0f14833 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:13.212781 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f42ba-fdb4-4fda-a71b-eb0e8a34e40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:13.212781 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f42ba-fdb4-4fda-a71b-eb0e8a34e40f None admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:13.212781 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f42ba-fdb4-4fda-a71b-eb0e8a34e40f None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:13.212781 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f42ba-fdb4-4fda-a71b-eb0e8a34e40f None admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:13.213197 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344f42ba-fdb4-4fda-a71b-eb0e8a34e40f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:13.213197 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344f42ba-fdb4-4fda-a71b-eb0e8a34e40f None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:13.245173 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:13.310937 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cb9ed7-4e3f-4e7b-af7b-1364d0f14833 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:13.341195 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 70/148] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Aug 29 22:28:12 2022] POST /identity/v3/users => generated 283 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:28:13.352382 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566c8d32-0c2f-4494-8c3b-294bc94642e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:13.353727 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c8d32-0c2f-4494-8c3b-294bc94642e6 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:13.355230 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c8d32-0c2f-4494-8c3b-294bc94642e6 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:13.355230 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c8d32-0c2f-4494-8c3b-294bc94642e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:13.355570 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566c8d32-0c2f-4494-8c3b-294bc94642e6 None admin] RBAC: Adding query filter params (role.id=ae06bb653e3b4990a9ddaa1a649c7e06, scope.project.id=bd224ec63c3748aa90cf3e984e32fa3f, user.id=3a00a8e520484973b5ffa1769dcee1fc) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:13.356103 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566c8d32-0c2f-4494-8c3b-294bc94642e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:13.358031 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566c8d32-0c2f-4494-8c3b-294bc94642e6 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:13.365985 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 79/149] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:28:13 2022] GET /identity/v3/role_assignments?user.id=3a00a8e520484973b5ffa1769dcee1fc&scope.project.id=bd224ec63c3748aa90cf3e984e32fa3f&role.id=ae06bb653e3b4990a9ddaa1a649c7e06 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:13.503970 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f9cea-c896-4e96-9320-4e8f4761e414 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:13.504496 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f9cea-c896-4e96-9320-4e8f4761e414 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:13.504496 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f9cea-c896-4e96-9320-4e8f4761e414 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:13.506278 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 71/150] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:13.511736 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d4724-ab24-4a7e-bda2-9e91f605a40f None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:13.512343 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d4724-ab24-4a7e-bda2-9e91f605a40f None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:13.512862 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d4724-ab24-4a7e-bda2-9e91f605a40f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:13.541459 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-346d4724-ab24-4a7e-bda2-9e91f605a40f None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:13.544672 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-346d4724-ab24-4a7e-bda2-9e91f605a40f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:13.640482 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 80/151] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:13.941853 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f9cea-c896-4e96-9320-4e8f4761e414 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:14.059823 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d86f39-4b97-46f0-beb1-f2a12864d0d9 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:14.060196 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d86f39-4b97-46f0-beb1-f2a12864d0d9 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:14.060504 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d86f39-4b97-46f0-beb1-f2a12864d0d9 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:14.062191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 81/152] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:28:14.068194 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3b283-0d22-4f7f-aecf-30286cbf3db4 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:14.068474 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3b283-0d22-4f7f-aecf-30286cbf3db4 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:14.068770 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3b283-0d22-4f7f-aecf-30286cbf3db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:14.085597 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f9cea-c896-4e96-9320-4e8f4761e414 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:14.089564 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-1cc3b283-0d22-4f7f-aecf-30286cbf3db4 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:14.090762 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-1cc3b283-0d22-4f7f-aecf-30286cbf3db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:14.106539 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:14.130453 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:14.142380 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada1bbc-eece-442b-b73d-e7392d2bbe63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:14.143612 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada1bbc-eece-442b-b73d-e7392d2bbe63 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:14.143918 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada1bbc-eece-442b-b73d-e7392d2bbe63 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:14.144161 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada1bbc-eece-442b-b73d-e7392d2bbe63 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:14.169384 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ada1bbc-eece-442b-b73d-e7392d2bbe63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:14.174693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-0ada1bbc-eece-442b-b73d-e7392d2bbe63 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 22:28:14.174693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:14.174693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:14.174693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:14.174693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:14.174693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:14.174693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:14.174693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:14.174693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:14.174693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:14.174693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:14.174693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:14.174693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:14.174693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:14.174693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:14.174693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:14.179891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:14.179891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:14.179891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:14.179891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:14.179891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:14.179891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:14.179891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:14.179891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:14.179891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:14.179891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:14.179891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:14.179891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:14.179891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:14.179891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:14.179891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:14.179891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:14.180661 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:14.180661 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:14.180661 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:14.180661 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:14.180661 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:14.180661 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:14.180661 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:14.180661 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:14.180661 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:14.180661 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:14.180661 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 22:28:14.180661 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:28:14.180661 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 72/153] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:28:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:14.182007 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada1bbc-eece-442b-b73d-e7392d2bbe63 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:14.184981 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 82/154] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 22:28:14.334019 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada1bbc-eece-442b-b73d-e7392d2bbe63 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:14.347937 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5cbc9-7b59-4782-9237-1f90cfa00da3 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:14.348367 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5cbc9-7b59-4782-9237-1f90cfa00da3 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:14.349873 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5cbc9-7b59-4782-9237-1f90cfa00da3 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:14.351669 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 83/155] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:14.357444 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b84c8-74a9-4149-913a-c2a9593d4c10 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:14.358010 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b84c8-74a9-4149-913a-c2a9593d4c10 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:14.358170 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b84c8-74a9-4149-913a-c2a9593d4c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:14.385965 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-430b84c8-74a9-4149-913a-c2a9593d4c10 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:14.386948 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1aabb2-75be-467c-a919-ff4c3c4764b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:14.387158 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-430b84c8-74a9-4149-913a-c2a9593d4c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:14.388019 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1aabb2-75be-467c-a919-ff4c3c4764b5 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:14.388511 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1aabb2-75be-467c-a919-ff4c3c4764b5 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:14.389042 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1aabb2-75be-467c-a919-ff4c3c4764b5 None admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:14.389916 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1aabb2-75be-467c-a919-ff4c3c4764b5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:14.390412 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1aabb2-75be-467c-a919-ff4c3c4764b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:14.392009 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1aabb2-75be-467c-a919-ff4c3c4764b5 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:14.404829 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 73/156] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Aug 29 22:28:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:28:14.406400 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e0f5ae-8f42-46e8-b5b1-a6c6d9e47764 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:14.406885 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e0f5ae-8f42-46e8-b5b1-a6c6d9e47764 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:14.407376 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e0f5ae-8f42-46e8-b5b1-a6c6d9e47764 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:14.412896 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 74/157] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:14.414334 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e0f5ae-8f42-46e8-b5b1-a6c6d9e47764 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:14.474475 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:14.497407 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 84/158] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:14.499053 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d3766-a4d8-45ad-8d38-1e02064f04e0 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:14.499541 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d3766-a4d8-45ad-8d38-1e02064f04e0 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:14.500065 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d3766-a4d8-45ad-8d38-1e02064f04e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:14.527211 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-707d3766-a4d8-45ad-8d38-1e02064f04e0 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:14.528880 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-707d3766-a4d8-45ad-8d38-1e02064f04e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:14.567806 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e0f5ae-8f42-46e8-b5b1-a6c6d9e47764 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:14.621104 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 85/159] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:14.621704 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d3766-a4d8-45ad-8d38-1e02064f04e0 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:14.622726 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75936157-3d38-41cc-98de-b27d0c98f19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:14.623371 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75936157-3d38-41cc-98de-b27d0c98f19e None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:14.623651 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75936157-3d38-41cc-98de-b27d0c98f19e None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:14.623923 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75936157-3d38-41cc-98de-b27d0c98f19e None admin] PATH_INFO: `/v3/users/e3d531be164e4c969fd482fbc6c8c7f5` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:14.629524 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75936157-3d38-41cc-98de-b27d0c98f19e None admin] RBAC: Authorizing `identity:get_user(user_id=e3d531be164e4c969fd482fbc6c8c7f5)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:14.631012 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75936157-3d38-41cc-98de-b27d0c98f19e None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:14.632673 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 75/160] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:28:14 2022] GET /identity/v3/users/e3d531be164e4c969fd482fbc6c8c7f5 => generated 289 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:28:14.659420 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:14.662546 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75936157-3d38-41cc-98de-b27d0c98f19e None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:14.750425 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d3766-a4d8-45ad-8d38-1e02064f04e0 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:14.797413 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9722c6cb-358f-4c45-ae88-f75656f128f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:14.798323 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9722c6cb-358f-4c45-ae88-f75656f128f5 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:14.798599 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9722c6cb-358f-4c45-ae88-f75656f128f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:14.798991 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9722c6cb-358f-4c45-ae88-f75656f128f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:14.813825 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9722c6cb-358f-4c45-ae88-f75656f128f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:14.821354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-9722c6cb-358f-4c45-ae88-f75656f128f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:28:14.821354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:14.821354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:14.821354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:14.821354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:14.821354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:14.821354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:14.821354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:14.821354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:14.821354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:14.821354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:14.821354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:14.821354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:14.821354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:14.821354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:14.821354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:14.822226 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:14.822226 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:14.822226 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:14.822226 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:14.822226 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:14.822226 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:14.822226 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:14.822226 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:14.822226 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:14.822226 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:14.822226 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:14.822226 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:14.822226 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:14.822226 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:14.822226 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:14.822226 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:14.823042 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:14.823042 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:14.823042 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:14.823042 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:14.823042 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:14.823042 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:14.823042 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:14.823042 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:14.823042 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:14.823042 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:14.823042 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:28:14.823042 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:28:14.823042 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 86/161] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Aug 29 22:28:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:14.829352 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9722c6cb-358f-4c45-ae88-f75656f128f5 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:14.847116 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75936157-3d38-41cc-98de-b27d0c98f19e None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:14.851419 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:14.889508 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedef000-6baa-4a40-b301-ec4a925e770d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:14.890296 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedef000-6baa-4a40-b301-ec4a925e770d None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:14.890686 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedef000-6baa-4a40-b301-ec4a925e770d None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:14.891217 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedef000-6baa-4a40-b301-ec4a925e770d None admin] PATH_INFO: `/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:14.897579 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedef000-6baa-4a40-b301-ec4a925e770d None admin] RBAC: Authorizing `identity:get_project(project_id=bd224ec63c3748aa90cf3e984e32fa3f)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:14.899103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedef000-6baa-4a40-b301-ec4a925e770d None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:14.901047 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 76/162] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:28:14 2022] GET /identity/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:28:14.917615 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedef000-6baa-4a40-b301-ec4a925e770d None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:14.948425 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:14.967110 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9722c6cb-358f-4c45-ae88-f75656f128f5 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:15.008440 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4bbbd9-e424-48cc-ab53-afd5a31526b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:15.009087 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4bbbd9-e424-48cc-ab53-afd5a31526b4 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:15.009367 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4bbbd9-e424-48cc-ab53-afd5a31526b4 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:15.009940 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4bbbd9-e424-48cc-ab53-afd5a31526b4 None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:15.010622 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4bbbd9-e424-48cc-ab53-afd5a31526b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:15.010929 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4bbbd9-e424-48cc-ab53-afd5a31526b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:15.012656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4bbbd9-e424-48cc-ab53-afd5a31526b4 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:15.023482 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 87/163] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:28:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:28:15.024295 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4bbbd9-e424-48cc-ab53-afd5a31526b4 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:15.140806 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4bbbd9-e424-48cc-ab53-afd5a31526b4 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:15.179167 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53075f96-59cc-452a-9769-2aff82a40e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:15.179880 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53075f96-59cc-452a-9769-2aff82a40e1f None admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:15.180164 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53075f96-59cc-452a-9769-2aff82a40e1f None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:15.180438 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53075f96-59cc-452a-9769-2aff82a40e1f None admin] PATH_INFO: `/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f/users/e3d531be164e4c969fd482fbc6c8c7f5/roles/b9691ffb7caa429ab9a5876dd3dc2ba3` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:15.182937 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53075f96-59cc-452a-9769-2aff82a40e1f None admin] RBAC: Authorizing `identity:create_grant(project_id=bd224ec63c3748aa90cf3e984e32fa3f, user_id=e3d531be164e4c969fd482fbc6c8c7f5, role_id=b9691ffb7caa429ab9a5876dd3dc2ba3)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:15.184392 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53075f96-59cc-452a-9769-2aff82a40e1f None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:15.205147 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedef000-6baa-4a40-b301-ec4a925e770d None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:15.243659 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ddc681-5a45-4162-95d4-2de1cd7ea066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:15.244292 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ddc681-5a45-4162-95d4-2de1cd7ea066 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:15.244572 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ddc681-5a45-4162-95d4-2de1cd7ea066 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:15.244868 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ddc681-5a45-4162-95d4-2de1cd7ea066 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:15.256849 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ddc681-5a45-4162-95d4-2de1cd7ea066 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:15.266434 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-69ddc681-5a45-4162-95d4-2de1cd7ea066 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:28:15.266434 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:15.266434 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:15.266434 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:15.266434 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:15.266434 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:15.266434 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:15.266434 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:15.266434 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:15.266434 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:15.266434 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:15.266434 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:15.266434 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:15.266434 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:15.266434 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:15.266434 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:15.267327 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:15.267327 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:15.267327 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:15.267327 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:15.267327 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:15.267327 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:15.267327 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:15.267327 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:15.267327 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:15.267327 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:15.267327 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:15.267327 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:15.267327 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:15.267327 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:15.267327 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:15.267327 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:15.268325 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:15.268325 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:15.268325 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:15.268325 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:15.268325 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:15.268325 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:15.268325 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:15.268325 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:15.268325 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:15.268325 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:15.268325 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:28:15.268325 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:28:15.271269 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 88/164] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Aug 29 22:28:15 2022] PUT /identity/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f/users/e3d531be164e4c969fd482fbc6c8c7f5/roles/b9691ffb7caa429ab9a5876dd3dc2ba3 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:28:15.279511 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 77/165] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:28:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:28:15.279697 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53075f96-59cc-452a-9769-2aff82a40e1f None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:15.279798 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ddc681-5a45-4162-95d4-2de1cd7ea066 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:15.300599 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:15.320208 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:15.406560 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ddc681-5a45-4162-95d4-2de1cd7ea066 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:15.447888 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b27ad9-716e-4d32-8636-7fd302abf045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:15.448586 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b27ad9-716e-4d32-8636-7fd302abf045 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:15.448896 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b27ad9-716e-4d32-8636-7fd302abf045 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:15.449235 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b27ad9-716e-4d32-8636-7fd302abf045 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:15.461185 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b27ad9-716e-4d32-8636-7fd302abf045 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:15.466779 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-75b27ad9-716e-4d32-8636-7fd302abf045 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:28:15.466779 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:15.466779 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:15.466779 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:15.466779 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:15.466779 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:15.466779 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:15.466779 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:15.466779 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:15.466779 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:15.466779 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:15.466779 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:15.466779 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:15.466779 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:15.466779 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:15.466779 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:15.467746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:15.467746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:15.467746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:15.467746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:15.467746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:15.467746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:15.467746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:15.467746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:15.467746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:15.467746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:15.467746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:15.467746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:15.467746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:15.467746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:15.467746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:15.467746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:15.468520 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:15.468520 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:15.468520 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:15.468520 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:15.468520 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:15.468520 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:15.468520 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:15.468520 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:15.468520 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:15.468520 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:15.468520 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:28:15.468520 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:28:15.468520 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 78/166] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Aug 29 22:28:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:15.469695 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b27ad9-716e-4d32-8636-7fd302abf045 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:15.505599 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53075f96-59cc-452a-9769-2aff82a40e1f None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:15.568751 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1000c0-ebf1-4cac-85e5-253a53fead5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:15.568751 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1000c0-ebf1-4cac-85e5-253a53fead5b None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:15.568751 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1000c0-ebf1-4cac-85e5-253a53fead5b None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:15.568751 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1000c0-ebf1-4cac-85e5-253a53fead5b None admin] PATH_INFO: `/v3/users/9cc82f35b5c947928c9caa5d0b7e11fc` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:15.569164 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1000c0-ebf1-4cac-85e5-253a53fead5b None admin] RBAC: Authorizing `identity:get_user(user_id=9cc82f35b5c947928c9caa5d0b7e11fc)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:15.569633 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1000c0-ebf1-4cac-85e5-253a53fead5b None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:15.571957 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 89/167] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:28:15 2022] GET /identity/v3/users/9cc82f35b5c947928c9caa5d0b7e11fc => generated 247 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:15.573114 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1000c0-ebf1-4cac-85e5-253a53fead5b None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:15.635539 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:15.670977 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b27ad9-716e-4d32-8636-7fd302abf045 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:15.673631 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:15.704709 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1000c0-ebf1-4cac-85e5-253a53fead5b None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:15.755193 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e56aa44-57a5-433e-9ccf-b248d7c4d827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:15.756242 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e56aa44-57a5-433e-9ccf-b248d7c4d827 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:15.756967 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e56aa44-57a5-433e-9ccf-b248d7c4d827 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:15.757566 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e56aa44-57a5-433e-9ccf-b248d7c4d827 None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:15.758544 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e56aa44-57a5-433e-9ccf-b248d7c4d827 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:15.759150 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e56aa44-57a5-433e-9ccf-b248d7c4d827 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:15.759613 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b009b-f9a0-41e9-a54b-099cabe84ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:15.760338 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b009b-f9a0-41e9-a54b-099cabe84ba1 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:15.760702 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b009b-f9a0-41e9-a54b-099cabe84ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:15.761074 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b009b-f9a0-41e9-a54b-099cabe84ba1 None admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:15.761695 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e56aa44-57a5-433e-9ccf-b248d7c4d827 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:15.761848 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83b009b-f9a0-41e9-a54b-099cabe84ba1 None admin] RBAC: Adding query filter params (name=member) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:15.762250 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83b009b-f9a0-41e9-a54b-099cabe84ba1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:15.763739 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83b009b-f9a0-41e9-a54b-099cabe84ba1 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:15.773226 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 90/168] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:28:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:15.774574 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e56aa44-57a5-433e-9ccf-b248d7c4d827 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:15.782933 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 79/169] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Aug 29 22:28:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:15.784639 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b009b-f9a0-41e9-a54b-099cabe84ba1 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:15.948432 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e56aa44-57a5-433e-9ccf-b248d7c4d827 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:15.995560 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b009b-f9a0-41e9-a54b-099cabe84ba1 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:16.000937 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ebca26-fa45-4f1e-8e29-5dfe62b2049c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:16.003398 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ebca26-fa45-4f1e-8e29-5dfe62b2049c None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:16.003891 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ebca26-fa45-4f1e-8e29-5dfe62b2049c None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:16.004340 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ebca26-fa45-4f1e-8e29-5dfe62b2049c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:16.017454 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:16.090844 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f01561-c929-48f3-bbbc-1944424aea4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:16.091488 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f01561-c929-48f3-bbbc-1944424aea4e None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:16.091766 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f01561-c929-48f3-bbbc-1944424aea4e None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:16.092038 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f01561-c929-48f3-bbbc-1944424aea4e None admin] PATH_INFO: `/v3/users/3a00a8e520484973b5ffa1769dcee1fc` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:16.097484 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f01561-c929-48f3-bbbc-1944424aea4e None admin] RBAC: Authorizing `identity:get_user(user_id=3a00a8e520484973b5ffa1769dcee1fc)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:16.098997 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f01561-c929-48f3-bbbc-1944424aea4e None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:16.104926 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 80/170] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:28:15 2022] GET /identity/v3/users/3a00a8e520484973b5ffa1769dcee1fc => generated 275 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:16.105698 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f01561-c929-48f3-bbbc-1944424aea4e None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:16.160963 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:16.174435 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ebca26-fa45-4f1e-8e29-5dfe62b2049c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:16.176923 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ebca26-fa45-4f1e-8e29-5dfe62b2049c None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:16.328619 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f01561-c929-48f3-bbbc-1944424aea4e None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:16.383439 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 91/171] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:28:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 609 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:28:16.384101 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ebca26-fa45-4f1e-8e29-5dfe62b2049c None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:16.414344 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b6b52-73ae-4851-bdcb-03d059511c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:16.414997 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b6b52-73ae-4851-bdcb-03d059511c32 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:16.415283 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b6b52-73ae-4851-bdcb-03d059511c32 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:16.415560 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b6b52-73ae-4851-bdcb-03d059511c32 None admin] PATH_INFO: `/v3/users/3a00a8e520484973b5ffa1769dcee1fc` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:16.419774 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078b6b52-73ae-4851-bdcb-03d059511c32 None admin] RBAC: Authorizing `identity:get_user(user_id=3a00a8e520484973b5ffa1769dcee1fc)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:16.421281 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078b6b52-73ae-4851-bdcb-03d059511c32 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:16.423150 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 81/172] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:28:16 2022] GET /identity/v3/users/3a00a8e520484973b5ffa1769dcee1fc => generated 275 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:16.425347 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b6b52-73ae-4851-bdcb-03d059511c32 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:16.450495 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:16.523398 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ebca26-fa45-4f1e-8e29-5dfe62b2049c None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:16.590091 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae80223b-78f1-4f98-8917-34bf01492ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:16.590631 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80223b-78f1-4f98-8917-34bf01492ba5 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:16.590936 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80223b-78f1-4f98-8917-34bf01492ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:16.591212 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80223b-78f1-4f98-8917-34bf01492ba5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:16.694422 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:16.700440 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae80223b-78f1-4f98-8917-34bf01492ba5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:16.705686 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae80223b-78f1-4f98-8917-34bf01492ba5 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:16.730704 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b6b52-73ae-4851-bdcb-03d059511c32 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:16.765960 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:16.825463 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9681c-8e93-4fd8-828b-c3f3d76ea837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:16.826124 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9681c-8e93-4fd8-828b-c3f3d76ea837 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:16.826411 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9681c-8e93-4fd8-828b-c3f3d76ea837 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:16.826698 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9681c-8e93-4fd8-828b-c3f3d76ea837 None admin] PATH_INFO: `/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:16.827890 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a9681c-8e93-4fd8-828b-c3f3d76ea837 None admin] RBAC: Authorizing `identity:get_project(project_id=bd224ec63c3748aa90cf3e984e32fa3f)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:16.829400 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a9681c-8e93-4fd8-828b-c3f3d76ea837 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:16.831665 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 82/173] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:28:16 2022] GET /identity/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f => generated 298 bytes in 408 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:16.836332 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9681c-8e93-4fd8-828b-c3f3d76ea837 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:16.883276 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 92/174] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:28:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:28:16.884578 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae80223b-78f1-4f98-8917-34bf01492ba5 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:17.068247 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae80223b-78f1-4f98-8917-34bf01492ba5 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:17.092475 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9681c-8e93-4fd8-828b-c3f3d76ea837 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:17.112055 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2536b89-2dab-4ee0-902d-80484dcecff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:17.113542 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2536b89-2dab-4ee0-902d-80484dcecff0 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:17.114040 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2536b89-2dab-4ee0-902d-80484dcecff0 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:17.114505 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2536b89-2dab-4ee0-902d-80484dcecff0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:17.115425 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2536b89-2dab-4ee0-902d-80484dcecff0 None admin] RBAC: Adding query filter params (role.id=882f35bd847a406eb2222aeafc517a46, scope.project.id=bd224ec63c3748aa90cf3e984e32fa3f, user.id=9cc82f35b5c947928c9caa5d0b7e11fc) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:17.115943 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2536b89-2dab-4ee0-902d-80484dcecff0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:17.117974 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2536b89-2dab-4ee0-902d-80484dcecff0 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:17.125461 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 93/175] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:28:16 2022] GET /identity/v3/role_assignments?user.id=9cc82f35b5c947928c9caa5d0b7e11fc&scope.project.id=bd224ec63c3748aa90cf3e984e32fa3f&role.id=882f35bd847a406eb2222aeafc517a46 => generated 608 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:17.127463 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2536b89-2dab-4ee0-902d-80484dcecff0 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:17.130628 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:17.173139 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6683761d-3646-4855-a683-7ccdb6e6be1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:17.175570 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6683761d-3646-4855-a683-7ccdb6e6be1d None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:17.176104 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6683761d-3646-4855-a683-7ccdb6e6be1d None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:17.176577 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6683761d-3646-4855-a683-7ccdb6e6be1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:17.229316 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:17.301840 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6683761d-3646-4855-a683-7ccdb6e6be1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:17.303969 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6683761d-3646-4855-a683-7ccdb6e6be1d None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:17.344308 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2536b89-2dab-4ee0-902d-80484dcecff0 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:17.388332 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6947ef47-497a-4ef1-a991-d220bc065695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:17.389097 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947ef47-497a-4ef1-a991-d220bc065695 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:17.389441 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947ef47-497a-4ef1-a991-d220bc065695 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:17.389787 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947ef47-497a-4ef1-a991-d220bc065695 None admin] PATH_INFO: `/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:17.391006 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6947ef47-497a-4ef1-a991-d220bc065695 None admin] RBAC: Authorizing `identity:get_project(project_id=bd224ec63c3748aa90cf3e984e32fa3f)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:17.393252 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6947ef47-497a-4ef1-a991-d220bc065695 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:17.395163 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 94/176] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:28:17 2022] GET /identity/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f => generated 298 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:17.396507 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc22a4c-f287-467a-9ac1-1023b1c70464 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:17.396901 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc22a4c-f287-467a-9ac1-1023b1c70464 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:17.397248 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc22a4c-f287-467a-9ac1-1023b1c70464 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:17.398677 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 95/177] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:17.399919 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e65995-1826-4bba-90b8-d309a0686df3 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:17.400372 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e65995-1826-4bba-90b8-d309a0686df3 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:17.400750 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e65995-1826-4bba-90b8-d309a0686df3 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:17.402566 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 96/178] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:17.403261 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e65995-1826-4bba-90b8-d309a0686df3 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:17.438616 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 83/179] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:28:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 606 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:28:17.440915 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782c4f8-05d7-406a-a4dd-7d3f5dd7aacc None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:17.441386 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782c4f8-05d7-406a-a4dd-7d3f5dd7aacc None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:17.441870 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782c4f8-05d7-406a-a4dd-7d3f5dd7aacc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:17.467752 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-3782c4f8-05d7-406a-a4dd-7d3f5dd7aacc None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:17.469971 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-3782c4f8-05d7-406a-a4dd-7d3f5dd7aacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:17.492354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:17.547773 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:17.550925 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e65995-1826-4bba-90b8-d309a0686df3 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:17.571170 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 84/180] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:17.572287 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c919fb-9217-4922-a4c0-d8d564c4ea6d None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:17.573607 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c919fb-9217-4922-a4c0-d8d564c4ea6d None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:17.573607 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c919fb-9217-4922-a4c0-d8d564c4ea6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:17.591993 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc8f837-c9d3-4154-a475-05180dfe97b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:17.592985 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc8f837-c9d3-4154-a475-05180dfe97b1 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:17.593529 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc8f837-c9d3-4154-a475-05180dfe97b1 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:17.594012 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc8f837-c9d3-4154-a475-05180dfe97b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:17.595040 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc8f837-c9d3-4154-a475-05180dfe97b1 None admin] RBAC: Adding query filter params (role.id=a6d1677ebe3a4355be5d67c5c78d83c7, scope.project.id=bd224ec63c3748aa90cf3e984e32fa3f, user.id=3a00a8e520484973b5ffa1769dcee1fc) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:17.595146 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-29c919fb-9217-4922-a4c0-d8d564c4ea6d None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:17.596169 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc8f837-c9d3-4154-a475-05180dfe97b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:17.596834 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-29c919fb-9217-4922-a4c0-d8d564c4ea6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:17.598123 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc8f837-c9d3-4154-a475-05180dfe97b1 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:17.604834 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 97/181] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:28:17 2022] GET /identity/v3/role_assignments?user.id=3a00a8e520484973b5ffa1769dcee1fc&scope.project.id=bd224ec63c3748aa90cf3e984e32fa3f&role.id=a6d1677ebe3a4355be5d67c5c78d83c7 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:17.605980 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc8f837-c9d3-4154-a475-05180dfe97b1 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:17.756526 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 85/182] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:17.758567 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570f5bb-3942-42c1-bc69-327f1d08358e None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:17.760667 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570f5bb-3942-42c1-bc69-327f1d08358e None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:17.762313 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570f5bb-3942-42c1-bc69-327f1d08358e None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:17.763807 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 86/183] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:17.770391 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d028b2d-59cb-49c8-bfa7-3adb7349229e None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:17.770989 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d028b2d-59cb-49c8-bfa7-3adb7349229e None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:17.771514 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d028b2d-59cb-49c8-bfa7-3adb7349229e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:17.795985 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc8f837-c9d3-4154-a475-05180dfe97b1 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:17.809321 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-8d028b2d-59cb-49c8-bfa7-3adb7349229e None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:17.811340 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-8d028b2d-59cb-49c8-bfa7-3adb7349229e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:17.849854 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b000efc-3a07-42d1-bfda-888152811b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:17.850945 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b000efc-3a07-42d1-bfda-888152811b9c None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:17.851440 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b000efc-3a07-42d1-bfda-888152811b9c None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:17.851937 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b000efc-3a07-42d1-bfda-888152811b9c None admin] PATH_INFO: `/v3/projects/62d4a24220ec4112bc3143d0a99fa0e3` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:17.853335 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b000efc-3a07-42d1-bfda-888152811b9c None admin] RBAC: Authorizing `identity:get_project(project_id=62d4a24220ec4112bc3143d0a99fa0e3)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:17.855005 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b000efc-3a07-42d1-bfda-888152811b9c None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:17.857636 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 98/184] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:28:17 2022] GET /identity/v3/projects/62d4a24220ec4112bc3143d0a99fa0e3 => generated 312 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:17.858602 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b000efc-3a07-42d1-bfda-888152811b9c None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:17.861749 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:17.913556 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 87/185] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:17.914871 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d028b2d-59cb-49c8-bfa7-3adb7349229e None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:18.005789 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:18.009903 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b000efc-3a07-42d1-bfda-888152811b9c None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:18.052589 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f85945-f0fb-4394-84bf-2bf502046ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:18.053686 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f85945-f0fb-4394-84bf-2bf502046ce2 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:18.054238 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f85945-f0fb-4394-84bf-2bf502046ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:18.054807 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f85945-f0fb-4394-84bf-2bf502046ce2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:18.069419 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f85945-f0fb-4394-84bf-2bf502046ce2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:18.072551 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d028b2d-59cb-49c8-bfa7-3adb7349229e None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:18.076427 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-d1f85945-f0fb-4394-84bf-2bf502046ce2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:28:18.076427 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:18.076427 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:18.076427 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:18.076427 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:18.076427 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:18.076427 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:18.076427 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:18.076427 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:18.076427 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:18.076427 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:18.076427 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:18.076427 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:18.076427 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:18.076427 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:18.076427 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:18.077514 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:18.077514 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:18.077514 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:18.077514 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:18.077514 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:18.077514 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:18.077514 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:18.077514 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:18.077514 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:18.077514 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:18.077514 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:18.077514 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:18.077514 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:18.077514 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:18.077514 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:18.077514 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:18.078292 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:18.078292 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:18.078292 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:18.078292 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:18.078292 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:18.078292 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:18.078292 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:18.078292 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:18.078292 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:18.078292 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:18.078292 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:28:18.078292 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:28:18.081190 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 99/186] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Aug 29 22:28:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:18.088564 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f85945-f0fb-4394-84bf-2bf502046ce2 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:18.114166 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc92149-6201-4c34-9c1e-f072b1377000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:18.114860 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc92149-6201-4c34-9c1e-f072b1377000 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:18.115136 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc92149-6201-4c34-9c1e-f072b1377000 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:18.115406 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc92149-6201-4c34-9c1e-f072b1377000 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:18.116792 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc92149-6201-4c34-9c1e-f072b1377000 None admin] RBAC: Adding query filter params (role.id=ae06bb653e3b4990a9ddaa1a649c7e06, scope.project.id=62d4a24220ec4112bc3143d0a99fa0e3, user.id=3a00a8e520484973b5ffa1769dcee1fc) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:18.116792 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc92149-6201-4c34-9c1e-f072b1377000 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:18.118204 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc92149-6201-4c34-9c1e-f072b1377000 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:18.126643 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 88/187] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:28:17 2022] GET /identity/v3/role_assignments?user.id=3a00a8e520484973b5ffa1769dcee1fc&scope.project.id=62d4a24220ec4112bc3143d0a99fa0e3&role.id=ae06bb653e3b4990a9ddaa1a649c7e06 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:18.128170 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc92149-6201-4c34-9c1e-f072b1377000 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:18.163379 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:18.212662 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f85945-f0fb-4394-84bf-2bf502046ce2 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:18.251054 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a52d797-0ac2-4761-9889-47927b58559e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:18.252631 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52d797-0ac2-4761-9889-47927b58559e None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:18.253246 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52d797-0ac2-4761-9889-47927b58559e None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:18.253934 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52d797-0ac2-4761-9889-47927b58559e None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:18.254943 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a52d797-0ac2-4761-9889-47927b58559e None admin] RBAC: Adding query filter params (name=admin) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:18.255459 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a52d797-0ac2-4761-9889-47927b58559e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:18.257288 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a52d797-0ac2-4761-9889-47927b58559e None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:18.259553 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc92149-6201-4c34-9c1e-f072b1377000 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:18.266844 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 100/188] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:28:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:18.276342 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:18.278447 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a52d797-0ac2-4761-9889-47927b58559e None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:18.316752 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f238079f-53b3-4a39-be83-ef32ed49915a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:18.316752 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238079f-53b3-4a39-be83-ef32ed49915a None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:18.316752 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238079f-53b3-4a39-be83-ef32ed49915a None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:18.316752 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238079f-53b3-4a39-be83-ef32ed49915a None admin] PATH_INFO: `/v3/domains/default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:18.317257 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f238079f-53b3-4a39-be83-ef32ed49915a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:18.317257 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f238079f-53b3-4a39-be83-ef32ed49915a None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:18.326027 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 89/189] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:28:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:18.326027 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f238079f-53b3-4a39-be83-ef32ed49915a None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:18.435950 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a52d797-0ac2-4761-9889-47927b58559e None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:18.451606 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f238079f-53b3-4a39-be83-ef32ed49915a None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:18.461969 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:18.474294 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d09be6-864b-4c80-b78f-3296d78c6141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:18.474924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d09be6-864b-4c80-b78f-3296d78c6141 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:18.489273 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d09be6-864b-4c80-b78f-3296d78c6141 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:18.489273 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d09be6-864b-4c80-b78f-3296d78c6141 None admin] PATH_INFO: `/v3/users/c68c4a8dec564fac92e8cd76f5b062b6` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:18.490884 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d09be6-864b-4c80-b78f-3296d78c6141 None admin] RBAC: Authorizing `identity:get_user(user_id=c68c4a8dec564fac92e8cd76f5b062b6)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:18.492351 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d09be6-864b-4c80-b78f-3296d78c6141 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:18.492741 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c490f-da90-4a35-a681-3078c88ab6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:18.493373 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c490f-da90-4a35-a681-3078c88ab6f3 None admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:18.493655 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c490f-da90-4a35-a681-3078c88ab6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:18.494073 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 101/190] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:28:18 2022] GET /identity/v3/users/c68c4a8dec564fac92e8cd76f5b062b6 => generated 283 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:18.495107 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c490f-da90-4a35-a681-3078c88ab6f3 None admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:18.495273 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7ad74-8967-48eb-ab9e-00e2f8158be6 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:18.495523 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7ad74-8967-48eb-ab9e-00e2f8158be6 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:18.495731 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3c490f-da90-4a35-a681-3078c88ab6f3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:18.495821 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7ad74-8967-48eb-ab9e-00e2f8158be6 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:18.501280 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3c490f-da90-4a35-a681-3078c88ab6f3 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:18.501450 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 102/191] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:18.501450 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f7ad74-8967-48eb-ab9e-00e2f8158be6 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:18.588908 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 90/192] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Aug 29 22:28:18 2022] POST /identity/v3/users => generated 297 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:28:18.589748 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c490f-da90-4a35-a681-3078c88ab6f3 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:18.619936 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f7ad74-8967-48eb-ab9e-00e2f8158be6 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:18.633949 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:18.659197 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d257a7-db99-46bd-9e7e-e16f7531db44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:18.659841 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d257a7-db99-46bd-9e7e-e16f7531db44 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:18.660110 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d257a7-db99-46bd-9e7e-e16f7531db44 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:18.660383 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d257a7-db99-46bd-9e7e-e16f7531db44 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:18.673243 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d257a7-db99-46bd-9e7e-e16f7531db44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:18.679681 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-81d257a7-db99-46bd-9e7e-e16f7531db44 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:28:18.679681 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:18.679681 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:18.679681 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:18.679681 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:18.679681 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:18.679681 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:18.679681 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:18.679681 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:18.679681 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:18.679681 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:18.679681 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:18.679681 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:18.679681 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:18.679681 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:18.679681 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:18.685352 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:18.685352 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:18.685352 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:18.685352 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:18.685352 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:18.685352 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:18.685352 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:18.685352 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:18.685352 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:18.685352 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:18.685352 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:18.685352 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:18.685352 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:18.685352 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:18.685352 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:18.685352 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:18.686235 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:18.686235 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:18.686235 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:18.686235 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:18.686235 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:18.686235 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:18.686235 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:18.686235 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:18.686235 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:18.686235 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:18.686235 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:28:18.686235 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:28:18.686235 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 103/193] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Aug 29 22:28:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:18.686235 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a581fa98-2349-4d99-a89b-66c9996fdfa7 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:18.687119 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a581fa98-2349-4d99-a89b-66c9996fdfa7 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:18.687119 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a581fa98-2349-4d99-a89b-66c9996fdfa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:18.703546 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-a581fa98-2349-4d99-a89b-66c9996fdfa7 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:18.706292 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-a581fa98-2349-4d99-a89b-66c9996fdfa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:18.718570 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c490f-da90-4a35-a681-3078c88ab6f3 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:18.754028 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:18.762439 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c6453-12c4-46f0-90c5-88e1c5a02d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:18.763054 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c6453-12c4-46f0-90c5-88e1c5a02d1c None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:18.763338 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c6453-12c4-46f0-90c5-88e1c5a02d1c None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:18.769676 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c6453-12c4-46f0-90c5-88e1c5a02d1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:18.787696 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 104/194] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:18.788901 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a581fa98-2349-4d99-a89b-66c9996fdfa7 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:18.868257 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195c6453-12c4-46f0-90c5-88e1c5a02d1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:18.869818 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195c6453-12c4-46f0-90c5-88e1c5a02d1c None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:18.912763 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:18.927802 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a581fa98-2349-4d99-a89b-66c9996fdfa7 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:18.978156 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36617a7e-ff0a-4dd6-bc61-d2a1a93ead35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:18.978771 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36617a7e-ff0a-4dd6-bc61-d2a1a93ead35 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:18.979047 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36617a7e-ff0a-4dd6-bc61-d2a1a93ead35 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:18.979318 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36617a7e-ff0a-4dd6-bc61-d2a1a93ead35 None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:18.979985 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36617a7e-ff0a-4dd6-bc61-d2a1a93ead35 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:18.980287 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36617a7e-ff0a-4dd6-bc61-d2a1a93ead35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:18.981882 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36617a7e-ff0a-4dd6-bc61-d2a1a93ead35 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:18.994016 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 105/195] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:28:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:19.000861 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36617a7e-ff0a-4dd6-bc61-d2a1a93ead35 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:19.003555 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 91/196] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:28:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:28:19.009749 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c6453-12c4-46f0-90c5-88e1c5a02d1c None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:19.041109 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:19.111895 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36617a7e-ff0a-4dd6-bc61-d2a1a93ead35 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:19.117976 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c6453-12c4-46f0-90c5-88e1c5a02d1c None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:19.148781 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1d674d-dc28-41fb-8af0-042be863d9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:19.149426 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d674d-dc28-41fb-8af0-042be863d9ba None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:19.149708 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d674d-dc28-41fb-8af0-042be863d9ba None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:19.149982 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d674d-dc28-41fb-8af0-042be863d9ba None admin] PATH_INFO: `/v3/users/c68c4a8dec564fac92e8cd76f5b062b6` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:19.151124 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1d674d-dc28-41fb-8af0-042be863d9ba None admin] RBAC: Authorizing `identity:get_user(user_id=c68c4a8dec564fac92e8cd76f5b062b6)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:19.152736 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1d674d-dc28-41fb-8af0-042be863d9ba None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:19.154908 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 106/197] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:28:18 2022] GET /identity/v3/users/c68c4a8dec564fac92e8cd76f5b062b6 => generated 283 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:19.155580 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a77a53-7dbf-49d4-9aae-12c79284b487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:19.155740 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1d674d-dc28-41fb-8af0-042be863d9ba None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:19.156608 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a77a53-7dbf-49d4-9aae-12c79284b487 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:19.157004 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a77a53-7dbf-49d4-9aae-12c79284b487 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:19.157370 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a77a53-7dbf-49d4-9aae-12c79284b487 None admin] PATH_INFO: `/v3/projects/baba6492015a48bfbca2121065b933ab` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:19.158731 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a77a53-7dbf-49d4-9aae-12c79284b487 None admin] RBAC: Authorizing `identity:get_project(project_id=baba6492015a48bfbca2121065b933ab)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:19.160401 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a77a53-7dbf-49d4-9aae-12c79284b487 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:19.162458 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 92/198] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:28:19 2022] GET /identity/v3/projects/baba6492015a48bfbca2121065b933ab => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:19.163328 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a77a53-7dbf-49d4-9aae-12c79284b487 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:19.185077 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:19.268776 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1d674d-dc28-41fb-8af0-042be863d9ba None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:19.270545 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:19.302436 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a77a53-7dbf-49d4-9aae-12c79284b487 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:19.306017 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aff512-ecfc-4305-b0a7-eea3d5114503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:19.306813 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aff512-ecfc-4305-b0a7-eea3d5114503 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:19.307166 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aff512-ecfc-4305-b0a7-eea3d5114503 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:19.307508 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aff512-ecfc-4305-b0a7-eea3d5114503 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:19.320812 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aff512-ecfc-4305-b0a7-eea3d5114503 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:19.326428 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-77aff512-ecfc-4305-b0a7-eea3d5114503 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 22:28:19.326428 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:19.326428 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:19.326428 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:19.326428 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:19.326428 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:19.326428 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:19.326428 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:19.326428 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:19.326428 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:19.326428 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:19.326428 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:19.326428 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:19.326428 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:19.326428 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:19.326428 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:19.327299 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:19.327299 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:19.327299 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:19.327299 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:19.327299 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:19.327299 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:19.327299 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:19.327299 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:19.327299 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:19.327299 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:19.327299 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:19.327299 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:19.327299 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:19.327299 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:19.327299 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:19.327299 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:19.328071 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:19.328071 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:19.328071 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:19.328071 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:19.328071 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:19.328071 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:19.328071 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:19.328071 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:19.328071 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:19.328071 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:19.328071 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 22:28:19.328071 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:28:19.328071 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 107/199] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:28:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:19.328884 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aff512-ecfc-4305-b0a7-eea3d5114503 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:19.347638 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994bb6a0-1f13-46d6-8b87-5dabe186c121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:19.348782 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994bb6a0-1f13-46d6-8b87-5dabe186c121 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:19.349738 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994bb6a0-1f13-46d6-8b87-5dabe186c121 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:19.350162 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994bb6a0-1f13-46d6-8b87-5dabe186c121 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:19.444626 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aff512-ecfc-4305-b0a7-eea3d5114503 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:19.444852 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994bb6a0-1f13-46d6-8b87-5dabe186c121 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:19.446417 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994bb6a0-1f13-46d6-8b87-5dabe186c121 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:19.450122 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:19.483984 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c771f5-a9eb-4989-a6a3-ed50980386f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:19.485044 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c771f5-a9eb-4989-a6a3-ed50980386f4 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:19.485689 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c771f5-a9eb-4989-a6a3-ed50980386f4 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:19.487344 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c771f5-a9eb-4989-a6a3-ed50980386f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:19.488605 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c771f5-a9eb-4989-a6a3-ed50980386f4 None admin] RBAC: Adding query filter params (role.id=882f35bd847a406eb2222aeafc517a46, scope.project.id=baba6492015a48bfbca2121065b933ab, user.id=c68c4a8dec564fac92e8cd76f5b062b6) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:19.489138 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c771f5-a9eb-4989-a6a3-ed50980386f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:19.490915 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c771f5-a9eb-4989-a6a3-ed50980386f4 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:19.497193 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 108/200] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:28:19 2022] GET /identity/v3/role_assignments?user.id=c68c4a8dec564fac92e8cd76f5b062b6&scope.project.id=baba6492015a48bfbca2121065b933ab&role.id=882f35bd847a406eb2222aeafc517a46 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:19.498227 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c771f5-a9eb-4989-a6a3-ed50980386f4 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:19.532830 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:19.602278 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 93/201] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:28:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:28:19.608533 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994bb6a0-1f13-46d6-8b87-5dabe186c121 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:19.714914 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c771f5-a9eb-4989-a6a3-ed50980386f4 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:19.760839 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994bb6a0-1f13-46d6-8b87-5dabe186c121 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:19.763464 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:19.771431 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed3eb57-5c9d-43f3-84a4-6f1c7b1b4263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:19.776223 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed3eb57-5c9d-43f3-84a4-6f1c7b1b4263 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:19.776223 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed3eb57-5c9d-43f3-84a4-6f1c7b1b4263 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:19.776223 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed3eb57-5c9d-43f3-84a4-6f1c7b1b4263 None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:19.776223 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed3eb57-5c9d-43f3-84a4-6f1c7b1b4263 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:19.776223 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed3eb57-5c9d-43f3-84a4-6f1c7b1b4263 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:19.776223 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed3eb57-5c9d-43f3-84a4-6f1c7b1b4263 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:19.795039 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 109/202] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Aug 29 22:28:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:19.806203 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed3eb57-5c9d-43f3-84a4-6f1c7b1b4263 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:19.813426 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f608019f-2c97-4028-8eab-834b86374f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:19.814066 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608019f-2c97-4028-8eab-834b86374f89 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:19.814886 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608019f-2c97-4028-8eab-834b86374f89 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:19.815172 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608019f-2c97-4028-8eab-834b86374f89 None admin] PATH_INFO: `/v3/projects/baba6492015a48bfbca2121065b933ab` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:19.817082 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f608019f-2c97-4028-8eab-834b86374f89 None admin] RBAC: Authorizing `identity:get_project(project_id=baba6492015a48bfbca2121065b933ab)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:19.819716 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f608019f-2c97-4028-8eab-834b86374f89 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:19.821407 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 94/203] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:28:19 2022] GET /identity/v3/projects/baba6492015a48bfbca2121065b933ab => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:28:19.830100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f608019f-2c97-4028-8eab-834b86374f89 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:19.942549 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:19.956578 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed3eb57-5c9d-43f3-84a4-6f1c7b1b4263 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:20.014643 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb1852e-1c9e-44a5-91fc-d38bbf33727c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:20.014643 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb1852e-1c9e-44a5-91fc-d38bbf33727c None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:20.014928 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb1852e-1c9e-44a5-91fc-d38bbf33727c None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:20.015028 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb1852e-1c9e-44a5-91fc-d38bbf33727c None admin] PATH_INFO: `/v3/users/e3d531be164e4c969fd482fbc6c8c7f5` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:20.017464 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb1852e-1c9e-44a5-91fc-d38bbf33727c None admin] RBAC: Authorizing `identity:get_user(user_id=e3d531be164e4c969fd482fbc6c8c7f5)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:20.019070 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb1852e-1c9e-44a5-91fc-d38bbf33727c None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:20.020678 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 110/204] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:28:19 2022] GET /identity/v3/users/e3d531be164e4c969fd482fbc6c8c7f5 => generated 289 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:28:20.028755 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb1852e-1c9e-44a5-91fc-d38bbf33727c None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:20.059093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f608019f-2c97-4028-8eab-834b86374f89 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:20.125679 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa15787-1b72-4fa6-928d-9ea34b23013b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:20.126291 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa15787-1b72-4fa6-928d-9ea34b23013b None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:20.126572 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa15787-1b72-4fa6-928d-9ea34b23013b None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:20.126894 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa15787-1b72-4fa6-928d-9ea34b23013b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:20.127838 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa15787-1b72-4fa6-928d-9ea34b23013b None admin] RBAC: Adding query filter params (role.id=a6d1677ebe3a4355be5d67c5c78d83c7, scope.project.id=baba6492015a48bfbca2121065b933ab, user.id=c68c4a8dec564fac92e8cd76f5b062b6) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:20.128152 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa15787-1b72-4fa6-928d-9ea34b23013b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:20.129674 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa15787-1b72-4fa6-928d-9ea34b23013b None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:20.139734 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 95/205] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:28:19 2022] GET /identity/v3/role_assignments?user.id=c68c4a8dec564fac92e8cd76f5b062b6&scope.project.id=baba6492015a48bfbca2121065b933ab&role.id=a6d1677ebe3a4355be5d67c5c78d83c7 => generated 259 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:20.187345 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb1852e-1c9e-44a5-91fc-d38bbf33727c None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:20.243348 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f840a4ad-4a36-4011-bd16-704479ce53ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:20.243899 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f840a4ad-4a36-4011-bd16-704479ce53ed None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:20.244181 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f840a4ad-4a36-4011-bd16-704479ce53ed None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:20.244453 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f840a4ad-4a36-4011-bd16-704479ce53ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:20.295653 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:20.357721 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f840a4ad-4a36-4011-bd16-704479ce53ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:20.359209 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f840a4ad-4a36-4011-bd16-704479ce53ed None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:20.484545 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 111/206] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:28:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:28:20.491227 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa15787-1b72-4fa6-928d-9ea34b23013b None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:20.529272 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:20.600794 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa15787-1b72-4fa6-928d-9ea34b23013b None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:20.642070 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f38aa4-dd4e-4166-a61d-704877a2fd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:20.642815 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f38aa4-dd4e-4166-a61d-704877a2fd68 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:20.643409 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f38aa4-dd4e-4166-a61d-704877a2fd68 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:20.643409 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f38aa4-dd4e-4166-a61d-704877a2fd68 None admin] PATH_INFO: `/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:20.644596 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f38aa4-dd4e-4166-a61d-704877a2fd68 None admin] RBAC: Authorizing `identity:get_project(project_id=bd224ec63c3748aa90cf3e984e32fa3f)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:20.646201 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f38aa4-dd4e-4166-a61d-704877a2fd68 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:20.648161 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 96/207] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:28:20 2022] GET /identity/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:20.654003 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f840a4ad-4a36-4011-bd16-704479ce53ed None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:20.730567 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:20.783033 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f840a4ad-4a36-4011-bd16-704479ce53ed None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:20.834155 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35cf52e-a7d5-47ba-8255-4ff5dbcfff18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:20.834780 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35cf52e-a7d5-47ba-8255-4ff5dbcfff18 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:20.835129 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35cf52e-a7d5-47ba-8255-4ff5dbcfff18 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:20.835407 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35cf52e-a7d5-47ba-8255-4ff5dbcfff18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:20.836153 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35cf52e-a7d5-47ba-8255-4ff5dbcfff18 None admin] RBAC: Adding query filter params (role.id=b9691ffb7caa429ab9a5876dd3dc2ba3, scope.project.id=bd224ec63c3748aa90cf3e984e32fa3f, user.id=e3d531be164e4c969fd482fbc6c8c7f5) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:20.836461 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35cf52e-a7d5-47ba-8255-4ff5dbcfff18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:20.837970 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35cf52e-a7d5-47ba-8255-4ff5dbcfff18 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:20.850797 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 112/208] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:28:20 2022] GET /identity/v3/role_assignments?user.id=e3d531be164e4c969fd482fbc6c8c7f5&scope.project.id=bd224ec63c3748aa90cf3e984e32fa3f&role.id=b9691ffb7caa429ab9a5876dd3dc2ba3 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:21.149456 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f9b34-1e3c-4b47-86bd-2cfa545d8102 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:21.149947 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f9b34-1e3c-4b47-86bd-2cfa545d8102 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:21.149947 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f9b34-1e3c-4b47-86bd-2cfa545d8102 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:21.151522 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 97/209] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:21.157442 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1937dba3-33ca-4823-a0ca-fdd8bbac3a43 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:21.157735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1937dba3-33ca-4823-a0ca-fdd8bbac3a43 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:21.158014 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1937dba3-33ca-4823-a0ca-fdd8bbac3a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:21.180450 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-1937dba3-33ca-4823-a0ca-fdd8bbac3a43 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:21.181660 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-1937dba3-33ca-4823-a0ca-fdd8bbac3a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:21.270465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 113/210] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:21.392158 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8248d6-19ee-4333-bfe2-3ea40a601fa6 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:21.392566 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8248d6-19ee-4333-bfe2-3ea40a601fa6 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:21.393370 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8248d6-19ee-4333-bfe2-3ea40a601fa6 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:21.395127 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 98/211] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:21.401167 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e623343-eb59-4251-a867-dfb199b7232c None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:21.401398 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e623343-eb59-4251-a867-dfb199b7232c None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:21.401683 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e623343-eb59-4251-a867-dfb199b7232c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:21.420341 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-2e623343-eb59-4251-a867-dfb199b7232c None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:21.421564 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-2e623343-eb59-4251-a867-dfb199b7232c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:21.474165 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:21.495594 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 114/212] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:21.556597 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8248d6-19ee-4333-bfe2-3ea40a601fa6 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:21.726715 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077c57b-e874-402c-b3cb-cd57428b20fa None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:21.727185 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077c57b-e874-402c-b3cb-cd57428b20fa None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:21.727843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077c57b-e874-402c-b3cb-cd57428b20fa None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:21.731193 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 115/213] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:21.735260 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8248d6-19ee-4333-bfe2-3ea40a601fa6 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:21.738033 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2356ff5d-6628-4546-9824-c41e914d81e3 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:21.738650 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2356ff5d-6628-4546-9824-c41e914d81e3 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:21.739142 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2356ff5d-6628-4546-9824-c41e914d81e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:21.740117 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:21.763008 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-2356ff5d-6628-4546-9824-c41e914d81e3 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:21.765350 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-2356ff5d-6628-4546-9824-c41e914d81e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:21.787373 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad765d-5e06-4611-8d5d-51cd0cf6892c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:21.788010 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad765d-5e06-4611-8d5d-51cd0cf6892c None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:21.788293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad765d-5e06-4611-8d5d-51cd0cf6892c None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:21.788609 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad765d-5e06-4611-8d5d-51cd0cf6892c None admin] PATH_INFO: `/v3/roles/member` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:21.801250 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ad765d-5e06-4611-8d5d-51cd0cf6892c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:21.806838 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-98ad765d-5e06-4611-8d5d-51cd0cf6892c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:28:21.806838 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:21.806838 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:21.806838 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:21.806838 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:21.806838 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:21.806838 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:21.806838 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:21.806838 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:21.806838 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:21.806838 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:21.806838 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:21.806838 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:21.806838 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:21.806838 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:21.806838 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:21.807709 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:21.807709 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:21.807709 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:21.807709 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:21.807709 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:21.807709 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:21.807709 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:21.807709 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:21.807709 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:21.807709 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:21.807709 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:21.807709 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:21.807709 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:21.807709 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:21.807709 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:21.807709 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:21.808492 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:21.808492 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:21.808492 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:21.808492 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:21.808492 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:21.808492 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:21.808492 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:21.808492 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:21.808492 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:21.808492 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:21.808492 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:28:21.808492 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:28:21.808492 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 99/214] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:28:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:21.813458 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad765d-5e06-4611-8d5d-51cd0cf6892c None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:21.852773 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 116/215] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:21.871826 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2356ff5d-6628-4546-9824-c41e914d81e3 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:21.944658 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad765d-5e06-4611-8d5d-51cd0cf6892c None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:21.987109 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce29bc5-73fc-4cf1-bced-0b62c9e7c22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:21.988102 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce29bc5-73fc-4cf1-bced-0b62c9e7c22c None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:21.988518 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce29bc5-73fc-4cf1-bced-0b62c9e7c22c None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:21.988963 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce29bc5-73fc-4cf1-bced-0b62c9e7c22c None admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:21.989889 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce29bc5-73fc-4cf1-bced-0b62c9e7c22c None admin] RBAC: Adding query filter params (name=member) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:21.990448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce29bc5-73fc-4cf1-bced-0b62c9e7c22c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:21.991829 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce29bc5-73fc-4cf1-bced-0b62c9e7c22c None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:21.999618 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2356ff5d-6628-4546-9824-c41e914d81e3 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:22.001128 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 100/216] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Aug 29 22:28:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:28:22.007283 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce29bc5-73fc-4cf1-bced-0b62c9e7c22c None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:22.036472 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:22.051248 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:22.089186 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f333d60-0388-49c6-be7b-4b96813842b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:22.089830 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f333d60-0388-49c6-be7b-4b96813842b6 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:22.090316 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f333d60-0388-49c6-be7b-4b96813842b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:22.090586 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f333d60-0388-49c6-be7b-4b96813842b6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:22.093555 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f333d60-0388-49c6-be7b-4b96813842b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:22.095001 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f333d60-0388-49c6-be7b-4b96813842b6 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:22.097236 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 117/217] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:28:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:28:22.102592 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f333d60-0388-49c6-be7b-4b96813842b6 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:22.125013 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce29bc5-73fc-4cf1-bced-0b62c9e7c22c None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:22.161358 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d8742-9fd7-42a1-9e6e-d5872c4e6e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:22.162027 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d8742-9fd7-42a1-9e6e-d5872c4e6e0a None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:22.162591 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d8742-9fd7-42a1-9e6e-d5872c4e6e0a None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:22.162591 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d8742-9fd7-42a1-9e6e-d5872c4e6e0a None admin] PATH_INFO: `/v3/users/2a3ac91cc6fe477aa8539ad27453b248` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:22.177544 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5d8742-9fd7-42a1-9e6e-d5872c4e6e0a None admin] RBAC: Authorizing `identity:get_user(user_id=2a3ac91cc6fe477aa8539ad27453b248)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:22.179019 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5d8742-9fd7-42a1-9e6e-d5872c4e6e0a None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:22.180861 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 101/218] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:28:22 2022] GET /identity/v3/users/2a3ac91cc6fe477aa8539ad27453b248 => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:28:22.181456 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d8742-9fd7-42a1-9e6e-d5872c4e6e0a None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:22.218676 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f333d60-0388-49c6-be7b-4b96813842b6 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:22.258500 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e5934-8194-4ab3-9bff-e567a22b1613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:22.259037 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e5934-8194-4ab3-9bff-e567a22b1613 None admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:22.259312 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e5934-8194-4ab3-9bff-e567a22b1613 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:22.259584 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e5934-8194-4ab3-9bff-e567a22b1613 None admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:22.260232 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61e5934-8194-4ab3-9bff-e567a22b1613 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:22.261779 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61e5934-8194-4ab3-9bff-e567a22b1613 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:22.274821 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:22.290613 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d8742-9fd7-42a1-9e6e-d5872c4e6e0a None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:22.324690 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb8269-42bb-44da-8e02-77fb06bdea1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:22.325368 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb8269-42bb-44da-8e02-77fb06bdea1a None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:22.325645 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb8269-42bb-44da-8e02-77fb06bdea1a None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:22.325908 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb8269-42bb-44da-8e02-77fb06bdea1a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:22.336847 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fb8269-42bb-44da-8e02-77fb06bdea1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:22.341746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-51fb8269-42bb-44da-8e02-77fb06bdea1a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:28:22.341746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:22.341746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:22.341746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:22.341746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:22.341746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:22.341746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:22.341746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:22.341746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:22.341746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:22.341746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:22.341746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:22.341746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:22.341746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:22.341746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:22.341746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:22.342854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:22.342854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:22.342854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:22.342854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:22.342854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:22.342854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:22.342854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:22.342854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:22.342854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:22.342854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:22.342854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:22.342854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:22.342854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:22.342854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:22.342854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:22.342854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:22.343677 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:22.343677 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:22.343677 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:22.343677 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:22.343677 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:22.343677 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:22.343677 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:22.343677 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:22.343677 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:22.343677 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:22.343677 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:28:22.343677 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:28:22.343677 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 102/219] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Aug 29 22:28:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:22.344334 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb8269-42bb-44da-8e02-77fb06bdea1a None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:22.368005 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 118/220] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Aug 29 22:28:22 2022] POST /identity/v3/users => generated 297 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:28:22.371568 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e5934-8194-4ab3-9bff-e567a22b1613 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:22.410798 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:22.489358 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e5934-8194-4ab3-9bff-e567a22b1613 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:22.490008 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb8269-42bb-44da-8e02-77fb06bdea1a None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:22.530071 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006fd3d4-5520-4c43-a327-2083f6ffda52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:22.530071 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006fd3d4-5520-4c43-a327-2083f6ffda52 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:22.530071 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006fd3d4-5520-4c43-a327-2083f6ffda52 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:22.530071 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006fd3d4-5520-4c43-a327-2083f6ffda52 None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:22.530458 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006fd3d4-5520-4c43-a327-2083f6ffda52 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:22.530706 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006fd3d4-5520-4c43-a327-2083f6ffda52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:22.533496 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006fd3d4-5520-4c43-a327-2083f6ffda52 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:22.539191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207199d0-7fe0-40ed-b9c3-c6ec672625fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:22.543724 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207199d0-7fe0-40ed-b9c3-c6ec672625fc None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:22.543724 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207199d0-7fe0-40ed-b9c3-c6ec672625fc None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:22.543724 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207199d0-7fe0-40ed-b9c3-c6ec672625fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:22.543996 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 119/221] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:28:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:22.545204 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6cea91-ff08-4814-857e-538f7bf58493 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:22.545204 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6cea91-ff08-4814-857e-538f7bf58493 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:22.545204 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6cea91-ff08-4814-857e-538f7bf58493 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:22.546419 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 120/222] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:22.548464 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6cea91-ff08-4814-857e-538f7bf58493 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:22.567093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:22.642344 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207199d0-7fe0-40ed-b9c3-c6ec672625fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:22.645527 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207199d0-7fe0-40ed-b9c3-c6ec672625fc None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:22.688802 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:22.697585 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6cea91-ff08-4814-857e-538f7bf58493 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:22.745405 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b395ea-991c-4edb-aeb6-fbc31b4cefae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:22.746205 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b395ea-991c-4edb-aeb6-fbc31b4cefae None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:22.746566 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b395ea-991c-4edb-aeb6-fbc31b4cefae None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:22.746915 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b395ea-991c-4edb-aeb6-fbc31b4cefae None admin] PATH_INFO: `/v3/users/c68c4a8dec564fac92e8cd76f5b062b6` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:22.748128 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b395ea-991c-4edb-aeb6-fbc31b4cefae None admin] RBAC: Authorizing `identity:get_user(user_id=c68c4a8dec564fac92e8cd76f5b062b6)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:22.749770 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b395ea-991c-4edb-aeb6-fbc31b4cefae None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:22.751541 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 121/223] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:28:22 2022] GET /identity/v3/users/c68c4a8dec564fac92e8cd76f5b062b6 => generated 283 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:28:22.753264 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d71d78-c997-4f9d-bbea-ef61b2f38e45 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:22.753665 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d71d78-c997-4f9d-bbea-ef61b2f38e45 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:22.753943 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d71d78-c997-4f9d-bbea-ef61b2f38e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:22.773922 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-93d71d78-c997-4f9d-bbea-ef61b2f38e45 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:22.775121 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-93d71d78-c997-4f9d-bbea-ef61b2f38e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:22.789306 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 103/224] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:28:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:28:22.790138 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207199d0-7fe0-40ed-b9c3-c6ec672625fc None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:22.843733 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:22.873292 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 122/225] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:22.874525 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d71d78-c997-4f9d-bbea-ef61b2f38e45 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:22.920947 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207199d0-7fe0-40ed-b9c3-c6ec672625fc None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:22.998166 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af1d3fa-6798-4714-b913-08ecbc9a19ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:22.998809 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af1d3fa-6798-4714-b913-08ecbc9a19ab None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:22.999088 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af1d3fa-6798-4714-b913-08ecbc9a19ab None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:22.999363 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af1d3fa-6798-4714-b913-08ecbc9a19ab None admin] PATH_INFO: `/v3/projects/baba6492015a48bfbca2121065b933ab` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:23.003470 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af1d3fa-6798-4714-b913-08ecbc9a19ab None admin] RBAC: Authorizing `identity:get_project(project_id=baba6492015a48bfbca2121065b933ab)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:23.006143 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af1d3fa-6798-4714-b913-08ecbc9a19ab None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:23.013503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 104/226] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:28:22 2022] GET /identity/v3/projects/baba6492015a48bfbca2121065b933ab => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:23.019669 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af1d3fa-6798-4714-b913-08ecbc9a19ab None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:23.050336 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:23.057454 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d71d78-c997-4f9d-bbea-ef61b2f38e45 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:23.097820 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32201d2-c7da-4c0d-babd-8562bdb0d363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:23.098507 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32201d2-c7da-4c0d-babd-8562bdb0d363 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:23.098793 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32201d2-c7da-4c0d-babd-8562bdb0d363 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:23.099068 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32201d2-c7da-4c0d-babd-8562bdb0d363 None admin] PATH_INFO: `/v3/projects/baba6492015a48bfbca2121065b933ab` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:23.100201 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32201d2-c7da-4c0d-babd-8562bdb0d363 None admin] RBAC: Authorizing `identity:get_project(project_id=baba6492015a48bfbca2121065b933ab)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:23.101764 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32201d2-c7da-4c0d-babd-8562bdb0d363 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:23.103539 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 123/227] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:28:22 2022] GET /identity/v3/projects/baba6492015a48bfbca2121065b933ab => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:23.109411 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32201d2-c7da-4c0d-babd-8562bdb0d363 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:23.223551 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af1d3fa-6798-4714-b913-08ecbc9a19ab None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:23.233382 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32201d2-c7da-4c0d-babd-8562bdb0d363 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:23.238699 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:23.262291 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cfea82-1428-4516-ba93-8da7ef49cef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:23.263138 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cfea82-1428-4516-ba93-8da7ef49cef8 None admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:23.263429 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cfea82-1428-4516-ba93-8da7ef49cef8 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:23.263707 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cfea82-1428-4516-ba93-8da7ef49cef8 None admin] PATH_INFO: `/v3/projects/baba6492015a48bfbca2121065b933ab/users/c68c4a8dec564fac92e8cd76f5b062b6/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:23.267513 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cfea82-1428-4516-ba93-8da7ef49cef8 None admin] RBAC: Authorizing `identity:create_grant(project_id=baba6492015a48bfbca2121065b933ab, user_id=c68c4a8dec564fac92e8cd76f5b062b6, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:23.269427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cfea82-1428-4516-ba93-8da7ef49cef8 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:23.274491 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aec293-9d11-4633-a2f8-5b82c1199143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:23.275273 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aec293-9d11-4633-a2f8-5b82c1199143 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:23.275688 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aec293-9d11-4633-a2f8-5b82c1199143 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:23.276094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aec293-9d11-4633-a2f8-5b82c1199143 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:23.277021 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aec293-9d11-4633-a2f8-5b82c1199143 None admin] RBAC: Adding query filter params (role.id=ae06bb653e3b4990a9ddaa1a649c7e06, scope.project.id=baba6492015a48bfbca2121065b933ab, user.id=2a3ac91cc6fe477aa8539ad27453b248) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:23.277523 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aec293-9d11-4633-a2f8-5b82c1199143 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:23.279073 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aec293-9d11-4633-a2f8-5b82c1199143 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:23.285074 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 124/228] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:28:23 2022] GET /identity/v3/role_assignments?user.id=2a3ac91cc6fe477aa8539ad27453b248&scope.project.id=baba6492015a48bfbca2121065b933ab&role.id=ae06bb653e3b4990a9ddaa1a649c7e06 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:28:23.286009 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aec293-9d11-4633-a2f8-5b82c1199143 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:23.320793 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:23.344281 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 105/229] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Aug 29 22:28:23 2022] PUT /identity/v3/projects/baba6492015a48bfbca2121065b933ab/users/c68c4a8dec564fac92e8cd76f5b062b6/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:28:23.406009 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aec293-9d11-4633-a2f8-5b82c1199143 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:23.445391 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fae66af-1dbd-4e2d-ad13-62d6c3d2f334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:23.446041 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fae66af-1dbd-4e2d-ad13-62d6c3d2f334 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:23.446316 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fae66af-1dbd-4e2d-ad13-62d6c3d2f334 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:23.446582 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fae66af-1dbd-4e2d-ad13-62d6c3d2f334 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:23.459854 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fae66af-1dbd-4e2d-ad13-62d6c3d2f334 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:23.465519 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-9fae66af-1dbd-4e2d-ad13-62d6c3d2f334 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:28:23.465519 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:23.465519 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:23.465519 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:23.465519 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:23.465519 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:23.465519 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:23.465519 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:23.465519 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:23.465519 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:23.465519 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:23.465519 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:23.465519 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:23.465519 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:23.465519 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:23.465519 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:23.466456 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:23.466456 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:23.466456 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:23.466456 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:23.466456 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:23.466456 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:23.466456 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:23.466456 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:23.466456 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:23.466456 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:23.466456 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:23.466456 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:23.466456 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:23.466456 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:23.466456 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:23.466456 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:23.467844 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:23.467844 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:23.467844 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:23.467844 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:23.467844 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:23.467844 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:23.467844 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:23.467844 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:23.467844 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:23.467844 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:23.467844 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:28:23.467844 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:28:23.467844 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 125/230] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Aug 29 22:28:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:23.473552 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cfea82-1428-4516-ba93-8da7ef49cef8 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:23.598699 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cfea82-1428-4516-ba93-8da7ef49cef8 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:23.637336 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c4b10-910b-4f53-a2e7-cdd5cd085f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:23.637961 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c4b10-910b-4f53-a2e7-cdd5cd085f17 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:23.638239 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c4b10-910b-4f53-a2e7-cdd5cd085f17 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:23.638514 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c4b10-910b-4f53-a2e7-cdd5cd085f17 None admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:23.639188 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374c4b10-910b-4f53-a2e7-cdd5cd085f17 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:23.639492 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374c4b10-910b-4f53-a2e7-cdd5cd085f17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:23.640897 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374c4b10-910b-4f53-a2e7-cdd5cd085f17 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:23.651110 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 106/231] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:28:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:23.656574 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fae66af-1dbd-4e2d-ad13-62d6c3d2f334 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:23.792000 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:23.809206 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fae66af-1dbd-4e2d-ad13-62d6c3d2f334 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:23.852362 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd267e44-1adb-4734-94e1-2d4356030470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:23.852941 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd267e44-1adb-4734-94e1-2d4356030470 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:23.853217 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd267e44-1adb-4734-94e1-2d4356030470 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:23.853491 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd267e44-1adb-4734-94e1-2d4356030470 None admin] PATH_INFO: `/v3/users/c68c4a8dec564fac92e8cd76f5b062b6` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:23.854630 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd267e44-1adb-4734-94e1-2d4356030470 None admin] RBAC: Authorizing `identity:get_user(user_id=c68c4a8dec564fac92e8cd76f5b062b6)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:23.856085 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd267e44-1adb-4734-94e1-2d4356030470 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:23.858010 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 126/232] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:28:23 2022] GET /identity/v3/users/c68c4a8dec564fac92e8cd76f5b062b6 => generated 283 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:28:23.863922 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c4b10-910b-4f53-a2e7-cdd5cd085f17 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:23.927841 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:24.020841 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c4b10-910b-4f53-a2e7-cdd5cd085f17 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:24.062030 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8bf21b-40f8-44df-8094-d9fefa5beb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:24.062675 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8bf21b-40f8-44df-8094-d9fefa5beb14 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:24.062955 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8bf21b-40f8-44df-8094-d9fefa5beb14 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:24.063228 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8bf21b-40f8-44df-8094-d9fefa5beb14 None admin] PATH_INFO: `/v3/projects/baba6492015a48bfbca2121065b933ab` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:24.064398 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8bf21b-40f8-44df-8094-d9fefa5beb14 None admin] RBAC: Authorizing `identity:get_project(project_id=baba6492015a48bfbca2121065b933ab)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:24.066180 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8bf21b-40f8-44df-8094-d9fefa5beb14 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:24.067849 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 107/233] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:28:23 2022] GET /identity/v3/projects/baba6492015a48bfbca2121065b933ab => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:24.073761 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd267e44-1adb-4734-94e1-2d4356030470 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:24.201215 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd267e44-1adb-4734-94e1-2d4356030470 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:24.249110 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc1948-03d0-4ac2-857e-80943492fe77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:24.249775 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc1948-03d0-4ac2-857e-80943492fe77 None admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:24.250058 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc1948-03d0-4ac2-857e-80943492fe77 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:24.250335 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc1948-03d0-4ac2-857e-80943492fe77 None admin] PATH_INFO: `/v3/projects/baba6492015a48bfbca2121065b933ab/users/c68c4a8dec564fac92e8cd76f5b062b6/roles/a6d1677ebe3a4355be5d67c5c78d83c7` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:24.254643 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cc1948-03d0-4ac2-857e-80943492fe77 None admin] RBAC: Authorizing `identity:create_grant(project_id=baba6492015a48bfbca2121065b933ab, user_id=c68c4a8dec564fac92e8cd76f5b062b6, role_id=a6d1677ebe3a4355be5d67c5c78d83c7)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:24.256115 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cc1948-03d0-4ac2-857e-80943492fe77 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:24.361979 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 127/234] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Aug 29 22:28:24 2022] PUT /identity/v3/projects/baba6492015a48bfbca2121065b933ab/users/c68c4a8dec564fac92e8cd76f5b062b6/roles/a6d1677ebe3a4355be5d67c5c78d83c7 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:28:24.831337 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52453ad7-ed4a-4d1e-8783-5c1091c3cf0e None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:24.832319 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52453ad7-ed4a-4d1e-8783-5c1091c3cf0e None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:24.832840 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52453ad7-ed4a-4d1e-8783-5c1091c3cf0e None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:24.834751 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 108/235] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:24.839846 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bddf05-3f32-4115-9c44-8ce8ed168899 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:24.840652 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bddf05-3f32-4115-9c44-8ce8ed168899 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:24.841167 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bddf05-3f32-4115-9c44-8ce8ed168899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:24.869726 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-e4bddf05-3f32-4115-9c44-8ce8ed168899 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:24.871253 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-e4bddf05-3f32-4115-9c44-8ce8ed168899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:24.883961 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:24.958815 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 128/236] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:25.003485 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86d4fb-a499-4a98-b79e-8f3947c3e326 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:25.004094 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86d4fb-a499-4a98-b79e-8f3947c3e326 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:25.004586 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86d4fb-a499-4a98-b79e-8f3947c3e326 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:25.006431 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 109/237] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:28:25.011728 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003e323-995d-490c-9d8c-f522a571e30c None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:25.012243 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003e323-995d-490c-9d8c-f522a571e30c None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:25.012739 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003e323-995d-490c-9d8c-f522a571e30c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:25.041079 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-4003e323-995d-490c-9d8c-f522a571e30c None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:25.042603 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-4003e323-995d-490c-9d8c-f522a571e30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:25.129279 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 129/238] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:25.256220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86d4fb-a499-4a98-b79e-8f3947c3e326 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:25.386429 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86d4fb-a499-4a98-b79e-8f3947c3e326 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:25.387958 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:25.432158 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec5c429-0703-4a60-a92b-9aa95484010e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:25.432807 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec5c429-0703-4a60-a92b-9aa95484010e None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:25.433086 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec5c429-0703-4a60-a92b-9aa95484010e None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:25.433355 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec5c429-0703-4a60-a92b-9aa95484010e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:25.449678 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec5c429-0703-4a60-a92b-9aa95484010e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:25.454501 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-3ec5c429-0703-4a60-a92b-9aa95484010e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 22:28:25.454501 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:25.454501 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:25.454501 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:25.454501 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:25.454501 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:25.454501 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:25.454501 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:25.454501 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:25.454501 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:25.454501 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:25.454501 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:25.454501 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:25.454501 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:25.454501 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:25.454501 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:25.455470 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:25.455470 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:25.455470 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:25.455470 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:25.455470 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:25.455470 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:25.455470 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:25.455470 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:25.455470 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:25.455470 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:25.455470 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:25.455470 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:25.455470 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:25.455470 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:25.455470 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:25.455470 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:25.456261 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:25.456261 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:25.456261 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:25.456261 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:25.456261 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:25.456261 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:25.456261 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:25.456261 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:25.456261 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:25.456261 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:25.456261 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 22:28:25.456261 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:28:25.456261 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 110/239] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:28:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:25.463805 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4003e323-995d-490c-9d8c-f522a571e30c None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:25.561797 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec5c429-0703-4a60-a92b-9aa95484010e None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:25.568078 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:25.599626 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4003e323-995d-490c-9d8c-f522a571e30c None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:25.637813 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e8630-ab8b-4b77-8609-059c2912f37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:25.638346 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e8630-ab8b-4b77-8609-059c2912f37d None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:25.638613 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e8630-ab8b-4b77-8609-059c2912f37d None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:25.638887 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e8630-ab8b-4b77-8609-059c2912f37d None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:25.639558 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83e8630-ab8b-4b77-8609-059c2912f37d None admin] RBAC: Adding query filter params (name=reader) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:25.639865 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83e8630-ab8b-4b77-8609-059c2912f37d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:25.641290 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83e8630-ab8b-4b77-8609-059c2912f37d None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:25.650461 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 130/240] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Aug 29 22:28:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:25.655738 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e8630-ab8b-4b77-8609-059c2912f37d None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:25.758741 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec5c429-0703-4a60-a92b-9aa95484010e None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:25.760149 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e8630-ab8b-4b77-8609-059c2912f37d None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:25.797820 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad444cb-0f2c-44b3-b4a3-9ffe658063c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:25.798454 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad444cb-0f2c-44b3-b4a3-9ffe658063c8 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:25.798778 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad444cb-0f2c-44b3-b4a3-9ffe658063c8 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:25.799204 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad444cb-0f2c-44b3-b4a3-9ffe658063c8 None admin] PATH_INFO: `/v3/users/22108af3965045308e960dd21fcf1d3f` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:25.807775 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:25.813866 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad444cb-0f2c-44b3-b4a3-9ffe658063c8 None admin] RBAC: Authorizing `identity:get_user(user_id=22108af3965045308e960dd21fcf1d3f)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:25.816018 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad444cb-0f2c-44b3-b4a3-9ffe658063c8 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:25.817848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 131/241] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:28:25 2022] GET /identity/v3/users/22108af3965045308e960dd21fcf1d3f => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:28:25.823314 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad444cb-0f2c-44b3-b4a3-9ffe658063c8 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:25.841196 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1528aa-85bc-4809-b86b-87980a22b27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:25.841851 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1528aa-85bc-4809-b86b-87980a22b27d None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:25.842136 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1528aa-85bc-4809-b86b-87980a22b27d None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:25.842412 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1528aa-85bc-4809-b86b-87980a22b27d None admin] PATH_INFO: `/v3/domains/default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:25.843603 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1528aa-85bc-4809-b86b-87980a22b27d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:25.845162 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1528aa-85bc-4809-b86b-87980a22b27d None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:25.847246 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 111/242] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:28:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:25.849354 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5537ecff-e2ad-4600-a9e4-bb12b93c2837 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:25.849640 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5537ecff-e2ad-4600-a9e4-bb12b93c2837 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:25.849918 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5537ecff-e2ad-4600-a9e4-bb12b93c2837 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:25.851273 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 112/243] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:25.857288 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf48043-d98b-46e4-90c5-641c13d8c7ad None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:25.857675 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf48043-d98b-46e4-90c5-641c13d8c7ad None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:25.857960 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf48043-d98b-46e4-90c5-641c13d8c7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:25.876669 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-eaf48043-d98b-46e4-90c5-641c13d8c7ad None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:25.878821 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-eaf48043-d98b-46e4-90c5-641c13d8c7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:25.905840 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:25.933185 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad444cb-0f2c-44b3-b4a3-9ffe658063c8 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:25.960898 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 113/244] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:25.960898 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf48043-d98b-46e4-90c5-641c13d8c7ad None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:25.975141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2304df4a-5f61-4443-8177-b299f975f0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:25.975969 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304df4a-5f61-4443-8177-b299f975f0d1 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:25.976512 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304df4a-5f61-4443-8177-b299f975f0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:25.976811 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304df4a-5f61-4443-8177-b299f975f0d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:26.062114 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:26.071948 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2304df4a-5f61-4443-8177-b299f975f0d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:26.073207 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2304df4a-5f61-4443-8177-b299f975f0d1 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:26.087676 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf48043-d98b-46e4-90c5-641c13d8c7ad None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:26.132796 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f734cd5-7399-416a-9a1a-9f14cdb53d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:26.133671 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f734cd5-7399-416a-9a1a-9f14cdb53d3e None admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:26.134020 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f734cd5-7399-416a-9a1a-9f14cdb53d3e None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:26.134361 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f734cd5-7399-416a-9a1a-9f14cdb53d3e None admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:26.135337 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f734cd5-7399-416a-9a1a-9f14cdb53d3e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:26.137578 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f734cd5-7399-416a-9a1a-9f14cdb53d3e None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:26.180200 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 132/245] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:28:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 22:28:26.181671 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3bdf6a-0ed9-442f-83a1-ed166e77097c None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:26.182166 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3bdf6a-0ed9-442f-83a1-ed166e77097c None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:26.182560 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3bdf6a-0ed9-442f-83a1-ed166e77097c None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:26.183928 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 133/246] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:26.185597 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3bdf6a-0ed9-442f-83a1-ed166e77097c None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:26.233418 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 114/247] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Aug 29 22:28:25 2022] POST /identity/v3/users => generated 293 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:28:26.238010 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88140071-5eb3-4393-88ba-731afe886252 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:26.238336 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88140071-5eb3-4393-88ba-731afe886252 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:26.240481 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88140071-5eb3-4393-88ba-731afe886252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:26.265313 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-88140071-5eb3-4393-88ba-731afe886252 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:26.266522 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-88140071-5eb3-4393-88ba-731afe886252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:26.279749 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:26.309051 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:26.319418 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3bdf6a-0ed9-442f-83a1-ed166e77097c None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:26.351319 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 115/248] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:26.352200 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88140071-5eb3-4393-88ba-731afe886252 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:26.359659 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671b13ad-22bb-48a5-8934-9d5559e844cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:26.360298 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b13ad-22bb-48a5-8934-9d5559e844cd None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:26.360631 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b13ad-22bb-48a5-8934-9d5559e844cd None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:26.360949 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b13ad-22bb-48a5-8934-9d5559e844cd None admin] PATH_INFO: `/v3/projects/baba6492015a48bfbca2121065b933ab` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:26.362238 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671b13ad-22bb-48a5-8934-9d5559e844cd None admin] RBAC: Authorizing `identity:get_project(project_id=baba6492015a48bfbca2121065b933ab)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:26.363794 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671b13ad-22bb-48a5-8934-9d5559e844cd None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:26.365881 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 134/249] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:28:26 2022] GET /identity/v3/projects/baba6492015a48bfbca2121065b933ab => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:26.373872 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671b13ad-22bb-48a5-8934-9d5559e844cd None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:26.470149 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88140071-5eb3-4393-88ba-731afe886252 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:26.544474 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671b13ad-22bb-48a5-8934-9d5559e844cd None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:26.562971 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e2a49e-5ac7-4cea-843f-6dba0062d3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:26.563969 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e2a49e-5ac7-4cea-843f-6dba0062d3f6 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:26.566456 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e2a49e-5ac7-4cea-843f-6dba0062d3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:26.567747 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e2a49e-5ac7-4cea-843f-6dba0062d3f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:26.586197 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e2a49e-5ac7-4cea-843f-6dba0062d3f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:26.594191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-01e2a49e-5ac7-4cea-843f-6dba0062d3f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:28:26.594191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:26.594191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:26.594191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:26.594191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:26.594191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:26.594191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:26.594191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:26.594191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:26.594191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:26.594191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:26.594191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:26.594191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:26.594191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:26.594191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:26.594191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:26.595079 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:26.595079 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:26.595079 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:26.595079 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:26.595079 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:26.595079 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:26.595079 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:26.595079 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:26.595079 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:26.595079 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:26.595079 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:26.595079 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:26.595079 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:26.595079 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:26.595079 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:26.595079 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:26.596987 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:26.596987 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:26.596987 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:26.596987 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:26.596987 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:26.596987 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:26.596987 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:26.596987 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:26.596987 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:26.596987 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:26.596987 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:28:26.596987 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:28:26.597585 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb2187c-5e3e-401a-a523-f81dca6100bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:26.597585 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb2187c-5e3e-401a-a523-f81dca6100bc None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:26.597585 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb2187c-5e3e-401a-a523-f81dca6100bc None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:26.597585 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb2187c-5e3e-401a-a523-f81dca6100bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:26.598380 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb2187c-5e3e-401a-a523-f81dca6100bc None admin] RBAC: Adding query filter params (role.id=b9691ffb7caa429ab9a5876dd3dc2ba3, scope.project.id=baba6492015a48bfbca2121065b933ab, user.id=22108af3965045308e960dd21fcf1d3f) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:26.598380 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb2187c-5e3e-401a-a523-f81dca6100bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:26.599766 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb2187c-5e3e-401a-a523-f81dca6100bc None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:26.600488 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 116/250] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Aug 29 22:28:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:26.609442 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e2a49e-5ac7-4cea-843f-6dba0062d3f6 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:26.611019 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 135/251] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:28:26 2022] GET /identity/v3/role_assignments?user.id=22108af3965045308e960dd21fcf1d3f&scope.project.id=baba6492015a48bfbca2121065b933ab&role.id=b9691ffb7caa429ab9a5876dd3dc2ba3 => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:26.623022 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:26.695805 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb2187c-5e3e-401a-a523-f81dca6100bc None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:26.748705 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:26.760092 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e2a49e-5ac7-4cea-843f-6dba0062d3f6 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:26.827196 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75cfa8a-a695-4c7a-8749-b52708975625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:26.827730 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75cfa8a-a695-4c7a-8749-b52708975625 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:26.828028 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75cfa8a-a695-4c7a-8749-b52708975625 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:26.828279 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75cfa8a-a695-4c7a-8749-b52708975625 None admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:26.829647 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75cfa8a-a695-4c7a-8749-b52708975625 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:26.831325 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75cfa8a-a695-4c7a-8749-b52708975625 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:26.832947 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75cfa8a-a695-4c7a-8749-b52708975625 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:26.842691 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb2187c-5e3e-401a-a523-f81dca6100bc None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:26.849856 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 117/252] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:28:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:28:26.851139 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb5ba1-07dd-4221-a629-47f6744a0cc1 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:26.851427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb5ba1-07dd-4221-a629-47f6744a0cc1 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:26.851704 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb5ba1-07dd-4221-a629-47f6744a0cc1 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:26.856780 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 118/253] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:26.858677 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb5ba1-07dd-4221-a629-47f6744a0cc1 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:26.896447 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a56ff3-4704-40e7-9490-5cbd5b8f50f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:26.897142 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a56ff3-4704-40e7-9490-5cbd5b8f50f5 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:26.897583 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a56ff3-4704-40e7-9490-5cbd5b8f50f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:26.897861 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a56ff3-4704-40e7-9490-5cbd5b8f50f5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:26.912487 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a56ff3-4704-40e7-9490-5cbd5b8f50f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:26.917968 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-a5a56ff3-4704-40e7-9490-5cbd5b8f50f5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:28:26.917968 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:26.917968 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:26.917968 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:26.917968 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:26.917968 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:26.917968 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:26.917968 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:26.917968 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:26.917968 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:26.917968 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:26.917968 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:26.917968 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:26.917968 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:26.917968 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:26.917968 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:26.918864 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:26.918864 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:26.918864 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:26.918864 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:26.918864 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:26.918864 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:26.918864 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:26.918864 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:26.918864 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:26.918864 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:26.918864 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:26.918864 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:26.918864 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:26.918864 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:26.918864 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:26.918864 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:26.919641 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:26.919641 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:26.919641 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:26.919641 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:26.919641 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:26.919641 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:26.919641 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:26.919641 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:26.919641 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:26.919641 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:26.919641 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:28:26.919641 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:28:26.919641 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 136/254] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:28:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:28:26.920288 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6fb6e6-0a5f-4492-b6ce-8121eaee585c None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:26.921146 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6fb6e6-0a5f-4492-b6ce-8121eaee585c None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:26.921146 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6fb6e6-0a5f-4492-b6ce-8121eaee585c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:26.953624 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-9e6fb6e6-0a5f-4492-b6ce-8121eaee585c None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:26.954845 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-9e6fb6e6-0a5f-4492-b6ce-8121eaee585c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:26.978593 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:27.040104 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb5ba1-07dd-4221-a629-47f6744a0cc1 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:27.086674 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 137/255] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:27.087570 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6fb6e6-0a5f-4492-b6ce-8121eaee585c None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:27.090579 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84399d20-5b24-4060-a915-fa858ee4632a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:27.091231 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84399d20-5b24-4060-a915-fa858ee4632a None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:27.091528 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84399d20-5b24-4060-a915-fa858ee4632a None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:27.091839 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84399d20-5b24-4060-a915-fa858ee4632a None admin] PATH_INFO: `/v3/users/c68c4a8dec564fac92e8cd76f5b062b6` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:27.094834 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84399d20-5b24-4060-a915-fa858ee4632a None admin] RBAC: Authorizing `identity:get_user(user_id=c68c4a8dec564fac92e8cd76f5b062b6)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:27.101411 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84399d20-5b24-4060-a915-fa858ee4632a None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:27.109124 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 119/256] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:28:26 2022] GET /identity/v3/users/c68c4a8dec564fac92e8cd76f5b062b6 => generated 283 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:28:27.110920 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84399d20-5b24-4060-a915-fa858ee4632a None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:27.140963 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:27.290365 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6fb6e6-0a5f-4492-b6ce-8121eaee585c None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:27.346684 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b675e3-322c-4c9d-a073-87a2e5394f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:27.347304 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b675e3-322c-4c9d-a073-87a2e5394f51 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:27.347580 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b675e3-322c-4c9d-a073-87a2e5394f51 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:27.347856 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b675e3-322c-4c9d-a073-87a2e5394f51 None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:27.348520 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b675e3-322c-4c9d-a073-87a2e5394f51 None admin] RBAC: Adding query filter params (name=member) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:27.350844 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84399d20-5b24-4060-a915-fa858ee4632a None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:27.352427 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b675e3-322c-4c9d-a073-87a2e5394f51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:27.353879 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b675e3-322c-4c9d-a073-87a2e5394f51 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:27.372067 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 138/257] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Aug 29 22:28:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:27.378793 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b675e3-322c-4c9d-a073-87a2e5394f51 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:27.417559 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05233838-aa79-41c9-80f7-0df92d7bb93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:27.418168 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05233838-aa79-41c9-80f7-0df92d7bb93d None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:27.418449 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05233838-aa79-41c9-80f7-0df92d7bb93d None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:27.418720 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05233838-aa79-41c9-80f7-0df92d7bb93d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:27.426821 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:27.497214 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:27.520387 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b675e3-322c-4c9d-a073-87a2e5394f51 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:27.566029 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c38515-66db-4d61-bbfb-84b22c0c37f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:27.566584 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c38515-66db-4d61-bbfb-84b22c0c37f2 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:27.566865 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c38515-66db-4d61-bbfb-84b22c0c37f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:27.567135 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c38515-66db-4d61-bbfb-84b22c0c37f2 None admin] PATH_INFO: `/v3/users/2a3ac91cc6fe477aa8539ad27453b248` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:27.568328 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c38515-66db-4d61-bbfb-84b22c0c37f2 None admin] RBAC: Authorizing `identity:get_user(user_id=2a3ac91cc6fe477aa8539ad27453b248)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:27.570164 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c38515-66db-4d61-bbfb-84b22c0c37f2 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:27.572331 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 139/258] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:28:27 2022] GET /identity/v3/users/2a3ac91cc6fe477aa8539ad27453b248 => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:27.581951 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c38515-66db-4d61-bbfb-84b22c0c37f2 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:27.584750 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05233838-aa79-41c9-80f7-0df92d7bb93d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:27.586496 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05233838-aa79-41c9-80f7-0df92d7bb93d None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:27.749712 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c38515-66db-4d61-bbfb-84b22c0c37f2 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:27.770357 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 120/259] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:28:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 660 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:28:27.771128 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05233838-aa79-41c9-80f7-0df92d7bb93d None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:27.812975 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e317ec15-14ac-4fc3-b59a-d676e001ba44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:27.813668 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e317ec15-14ac-4fc3-b59a-d676e001ba44 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:27.813954 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e317ec15-14ac-4fc3-b59a-d676e001ba44 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:27.814226 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e317ec15-14ac-4fc3-b59a-d676e001ba44 None admin] PATH_INFO: `/v3/projects/baba6492015a48bfbca2121065b933ab` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:27.816226 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e317ec15-14ac-4fc3-b59a-d676e001ba44 None admin] RBAC: Authorizing `identity:get_project(project_id=baba6492015a48bfbca2121065b933ab)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:27.817683 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e317ec15-14ac-4fc3-b59a-d676e001ba44 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:27.819353 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 140/260] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:28:27 2022] GET /identity/v3/projects/baba6492015a48bfbca2121065b933ab => generated 302 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:27.824247 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e317ec15-14ac-4fc3-b59a-d676e001ba44 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:27.849598 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:27.853610 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:27.971097 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05233838-aa79-41c9-80f7-0df92d7bb93d None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:28.003902 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e317ec15-14ac-4fc3-b59a-d676e001ba44 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:28.045661 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef5632c-6fe9-4573-bdad-cad5dcfb7c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:28.046308 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef5632c-6fe9-4573-bdad-cad5dcfb7c11 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:28.046585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef5632c-6fe9-4573-bdad-cad5dcfb7c11 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:28.046857 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef5632c-6fe9-4573-bdad-cad5dcfb7c11 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:28.067602 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef5632c-6fe9-4573-bdad-cad5dcfb7c11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:28.075637 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-6ef5632c-6fe9-4573-bdad-cad5dcfb7c11 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:28:28.075637 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:28.075637 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:28.075637 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:28.075637 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:28.075637 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:28.075637 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:28.075637 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:28.075637 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:28.075637 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:28.075637 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:28.075637 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:28.075637 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:28.075637 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:28.075637 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:28.075637 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:28.076574 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:28.076574 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:28.076574 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:28.076574 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:28.076574 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:28.076574 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:28.076574 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:28.076574 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:28.076574 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:28.076574 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:28.076574 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:28.076574 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:28.076574 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:28.076574 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:28.076574 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:28.076574 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:28.077614 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:28.077614 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:28.077614 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:28.077614 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:28.077614 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:28.077614 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:28.077614 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:28.077614 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:28.077614 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:28.077614 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:28.077614 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:28:28.077614 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:28:28.077614 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 121/261] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Aug 29 22:28:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:28.077614 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef5632c-6fe9-4573-bdad-cad5dcfb7c11 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:28.083911 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f56a029-7c22-4257-96f3-dfb9950bc606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:28.084540 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56a029-7c22-4257-96f3-dfb9950bc606 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:28.084845 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56a029-7c22-4257-96f3-dfb9950bc606 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:28.085204 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56a029-7c22-4257-96f3-dfb9950bc606 None admin] PATH_INFO: `/v3/projects/baba6492015a48bfbca2121065b933ab` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:28.086458 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f56a029-7c22-4257-96f3-dfb9950bc606 None admin] RBAC: Authorizing `identity:get_project(project_id=baba6492015a48bfbca2121065b933ab)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:28.087911 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f56a029-7c22-4257-96f3-dfb9950bc606 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:28.089582 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 141/262] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:28:27 2022] GET /identity/v3/projects/baba6492015a48bfbca2121065b933ab => generated 302 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:28.090341 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f56a029-7c22-4257-96f3-dfb9950bc606 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:28.245829 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:28.252155 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:28.254011 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef5632c-6fe9-4573-bdad-cad5dcfb7c11 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:28.273955 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f56a029-7c22-4257-96f3-dfb9950bc606 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:28.318347 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c876957f-8d0a-41bd-81bb-a0157625b0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:28.318935 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c876957f-8d0a-41bd-81bb-a0157625b0d0 None admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:28.319213 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c876957f-8d0a-41bd-81bb-a0157625b0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:28.319485 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c876957f-8d0a-41bd-81bb-a0157625b0d0 None admin] PATH_INFO: `/v3/projects/baba6492015a48bfbca2121065b933ab/users/2a3ac91cc6fe477aa8539ad27453b248/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:28.321738 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c876957f-8d0a-41bd-81bb-a0157625b0d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=baba6492015a48bfbca2121065b933ab, user_id=2a3ac91cc6fe477aa8539ad27453b248, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:28.323193 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c876957f-8d0a-41bd-81bb-a0157625b0d0 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:28.336649 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd25e02-9a36-42e6-91ca-ff436996375f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:28.337287 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd25e02-9a36-42e6-91ca-ff436996375f None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:28.338968 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd25e02-9a36-42e6-91ca-ff436996375f None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:28.338968 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd25e02-9a36-42e6-91ca-ff436996375f None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:28.338968 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd25e02-9a36-42e6-91ca-ff436996375f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:28.338968 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd25e02-9a36-42e6-91ca-ff436996375f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:28.340172 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd25e02-9a36-42e6-91ca-ff436996375f None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:28.349369 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 142/263] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:28:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:28.350188 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd25e02-9a36-42e6-91ca-ff436996375f None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:28.401998 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 122/264] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Aug 29 22:28:28 2022] PUT /identity/v3/projects/baba6492015a48bfbca2121065b933ab/users/2a3ac91cc6fe477aa8539ad27453b248/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:28:28.402596 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c876957f-8d0a-41bd-81bb-a0157625b0d0 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:28.531294 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd25e02-9a36-42e6-91ca-ff436996375f None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:28.606171 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760febb7-173e-4861-af38-aa8d0e70dcb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:28.606710 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760febb7-173e-4861-af38-aa8d0e70dcb2 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:28.606989 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760febb7-173e-4861-af38-aa8d0e70dcb2 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:28.607257 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760febb7-173e-4861-af38-aa8d0e70dcb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:28.607995 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760febb7-173e-4861-af38-aa8d0e70dcb2 None admin] RBAC: Adding query filter params (role.id=882f35bd847a406eb2222aeafc517a46, scope.project.id=baba6492015a48bfbca2121065b933ab, user.id=c68c4a8dec564fac92e8cd76f5b062b6) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:28.608297 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760febb7-173e-4861-af38-aa8d0e70dcb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:28.609786 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760febb7-173e-4861-af38-aa8d0e70dcb2 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:28.624970 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 143/265] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:28:28 2022] GET /identity/v3/role_assignments?user.id=c68c4a8dec564fac92e8cd76f5b062b6&scope.project.id=baba6492015a48bfbca2121065b933ab&role.id=882f35bd847a406eb2222aeafc517a46 => generated 608 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:28.628822 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c876957f-8d0a-41bd-81bb-a0157625b0d0 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:28.676072 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510bf4a-0893-48fd-a2b3-ee0844070a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:28.676818 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510bf4a-0893-48fd-a2b3-ee0844070a0e None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:28.677102 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510bf4a-0893-48fd-a2b3-ee0844070a0e None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:28.677379 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510bf4a-0893-48fd-a2b3-ee0844070a0e None admin] PATH_INFO: `/v3/users/c68c4a8dec564fac92e8cd76f5b062b6` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:28.678662 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5510bf4a-0893-48fd-a2b3-ee0844070a0e None admin] RBAC: Authorizing `identity:get_user(user_id=c68c4a8dec564fac92e8cd76f5b062b6)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:28.680215 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5510bf4a-0893-48fd-a2b3-ee0844070a0e None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:28.682174 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 123/266] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:28:28 2022] GET /identity/v3/users/c68c4a8dec564fac92e8cd76f5b062b6 => generated 283 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:28.688752 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760febb7-173e-4861-af38-aa8d0e70dcb2 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:28.702510 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:28.829026 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760febb7-173e-4861-af38-aa8d0e70dcb2 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:28.878974 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ce35d4-4924-44ff-b8ef-100c1e47001a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:28.879512 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce35d4-4924-44ff-b8ef-100c1e47001a None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:28.879798 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce35d4-4924-44ff-b8ef-100c1e47001a None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:28.880071 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce35d4-4924-44ff-b8ef-100c1e47001a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:28.969651 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ce35d4-4924-44ff-b8ef-100c1e47001a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:28.971864 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ce35d4-4924-44ff-b8ef-100c1e47001a None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:28.985548 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:29.098532 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 144/267] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:28:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:28:29.104947 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510bf4a-0893-48fd-a2b3-ee0844070a0e None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:29.126025 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c7999-9650-419c-b61b-3918bd4b671d None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:29.126343 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c7999-9650-419c-b61b-3918bd4b671d None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:29.126572 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c7999-9650-419c-b61b-3918bd4b671d None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:29.129321 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 145/268] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:29.140950 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1399c299-2bf8-4c74-a644-e2cb26d7286e None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:29.141243 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1399c299-2bf8-4c74-a644-e2cb26d7286e None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:29.141538 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1399c299-2bf8-4c74-a644-e2cb26d7286e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:29.143934 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:29.182006 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-1399c299-2bf8-4c74-a644-e2cb26d7286e None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:29.182965 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-1399c299-2bf8-4c74-a644-e2cb26d7286e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:29.274305 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510bf4a-0893-48fd-a2b3-ee0844070a0e None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:29.305067 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 146/269] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 22:28:29.306385 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5569f399-3c20-446d-9060-ec0839ee352a None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:29.306684 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5569f399-3c20-446d-9060-ec0839ee352a None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:29.306960 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5569f399-3c20-446d-9060-ec0839ee352a None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:29.308301 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 147/270] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:29.312356 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043281dc-c920-41ba-93b0-dc629346f923 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:29.312356 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043281dc-c920-41ba-93b0-dc629346f923 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:29.312356 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043281dc-c920-41ba-93b0-dc629346f923 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:29.315459 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 148/271] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:29.316665 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79951aba-0097-410e-b292-f5ac39a360a8 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:29.317004 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79951aba-0097-410e-b292-f5ac39a360a8 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:29.317274 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79951aba-0097-410e-b292-f5ac39a360a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:29.321411 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1680d42-e69b-4c8a-8696-c68fafde7ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:29.322254 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1680d42-e69b-4c8a-8696-c68fafde7ecf None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:29.322705 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1680d42-e69b-4c8a-8696-c68fafde7ecf None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:29.323147 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1680d42-e69b-4c8a-8696-c68fafde7ecf None admin] PATH_INFO: `/v3/projects/baba6492015a48bfbca2121065b933ab` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:29.325740 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1680d42-e69b-4c8a-8696-c68fafde7ecf None admin] RBAC: Authorizing `identity:get_project(project_id=baba6492015a48bfbca2121065b933ab)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:29.327390 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1680d42-e69b-4c8a-8696-c68fafde7ecf None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:29.329523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 124/272] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:28:29 2022] GET /identity/v3/projects/baba6492015a48bfbca2121065b933ab => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:29.332227 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb64a9e-8fe2-49fe-8b68-f36441e0090d None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:29.333054 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb64a9e-8fe2-49fe-8b68-f36441e0090d None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:29.333505 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb64a9e-8fe2-49fe-8b68-f36441e0090d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:29.336886 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-79951aba-0097-410e-b292-f5ac39a360a8 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:29.338051 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-79951aba-0097-410e-b292-f5ac39a360a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:29.355788 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-7bb64a9e-8fe2-49fe-8b68-f36441e0090d None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:29.357024 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-7bb64a9e-8fe2-49fe-8b68-f36441e0090d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:29.377109 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:29.427507 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 149/273] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:29.428020 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 125/274] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:29.429096 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79951aba-0097-410e-b292-f5ac39a360a8 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:29.585163 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79951aba-0097-410e-b292-f5ac39a360a8 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:29.623572 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcc4be2-e56c-4ae7-8dc7-6b27b340c278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:29.624438 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc4be2-e56c-4ae7-8dc7-6b27b340c278 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:29.624735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc4be2-e56c-4ae7-8dc7-6b27b340c278 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:29.625029 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc4be2-e56c-4ae7-8dc7-6b27b340c278 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:29.625763 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcc4be2-e56c-4ae7-8dc7-6b27b340c278 None admin] RBAC: Adding query filter params (role.id=a6d1677ebe3a4355be5d67c5c78d83c7, scope.project.id=baba6492015a48bfbca2121065b933ab, user.id=c68c4a8dec564fac92e8cd76f5b062b6) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:29.626071 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcc4be2-e56c-4ae7-8dc7-6b27b340c278 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:29.627897 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcc4be2-e56c-4ae7-8dc7-6b27b340c278 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:29.634986 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 150/275] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:28:29 2022] GET /identity/v3/role_assignments?user.id=c68c4a8dec564fac92e8cd76f5b062b6&scope.project.id=baba6492015a48bfbca2121065b933ab&role.id=a6d1677ebe3a4355be5d67c5c78d83c7 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:29.744240 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb64a9e-8fe2-49fe-8b68-f36441e0090d None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:29.760658 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcc4be2-e56c-4ae7-8dc7-6b27b340c278 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:29.780667 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:29.809423 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:29.878179 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb64a9e-8fe2-49fe-8b68-f36441e0090d None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:29.898477 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcc4be2-e56c-4ae7-8dc7-6b27b340c278 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:29.940965 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcdbc54-cb1c-4cf7-a1f6-7b5ec63f5fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:29.941612 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcdbc54-cb1c-4cf7-a1f6-7b5ec63f5fac None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:29.941891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcdbc54-cb1c-4cf7-a1f6-7b5ec63f5fac None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:29.942268 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcdbc54-cb1c-4cf7-a1f6-7b5ec63f5fac None admin] PATH_INFO: `/v3/roles/member` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:29.950997 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaaabe3-c211-4047-a41f-b95bf6f72710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:29.952016 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaaabe3-c211-4047-a41f-b95bf6f72710 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:29.952456 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaaabe3-c211-4047-a41f-b95bf6f72710 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:29.954941 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaaabe3-c211-4047-a41f-b95bf6f72710 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:29.961015 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcdbc54-cb1c-4cf7-a1f6-7b5ec63f5fac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:29.966365 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-cbcdbc54-cb1c-4cf7-a1f6-7b5ec63f5fac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:28:29.966365 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:29.966365 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:29.966365 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:29.966365 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:29.966365 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:29.966365 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:29.966365 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:29.966365 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:29.966365 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:29.966365 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:29.966365 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:29.966365 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:29.966365 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:29.966365 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:29.966365 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:29.967487 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:29.967487 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:29.967487 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:29.967487 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:29.967487 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:29.967487 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:29.967487 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:29.967487 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:29.967487 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:29.967487 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:29.967487 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:29.967487 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:29.967487 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:29.967487 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:29.967487 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:29.967487 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:29.968298 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:29.968298 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:29.968298 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:29.968298 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:29.968298 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:29.968298 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:29.968298 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:29.968298 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:29.968298 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:29.968298 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:29.968298 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:28:29.968298 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:28:29.969896 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaaabe3-c211-4047-a41f-b95bf6f72710 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:29.972178 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 126/276] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:28:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:29.973409 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcdbc54-cb1c-4cf7-a1f6-7b5ec63f5fac None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:29.976803 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-fbaaabe3-c211-4047-a41f-b95bf6f72710 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 22:28:29.976803 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:29.976803 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:29.976803 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:29.976803 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:29.976803 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:29.976803 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:29.976803 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:29.976803 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:29.976803 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:29.976803 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:29.976803 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:29.976803 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:29.976803 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:29.976803 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:29.976803 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:29.977691 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:29.977691 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:29.977691 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:29.977691 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:29.977691 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:29.977691 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:29.977691 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:29.977691 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:29.977691 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:29.977691 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:29.977691 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:29.977691 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:29.977691 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:29.977691 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:29.977691 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:29.977691 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:29.979261 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:29.979261 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:29.979261 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:29.979261 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:29.979261 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:29.979261 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:29.979261 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:29.979261 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:29.979261 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:29.979261 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:29.979261 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 22:28:29.979261 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:28:29.979261 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 151/277] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:28:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:29.979261 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaaabe3-c211-4047-a41f-b95bf6f72710 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:30.125929 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:30.135723 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaaabe3-c211-4047-a41f-b95bf6f72710 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:30.153396 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:30.172323 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbe42e5-eb96-4dbb-b15b-2f39762f7054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:30.173232 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe42e5-eb96-4dbb-b15b-2f39762f7054 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:30.173501 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe42e5-eb96-4dbb-b15b-2f39762f7054 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:30.173812 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe42e5-eb96-4dbb-b15b-2f39762f7054 None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:30.174769 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbe42e5-eb96-4dbb-b15b-2f39762f7054 None admin] RBAC: Adding query filter params (name=member) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:30.175146 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbe42e5-eb96-4dbb-b15b-2f39762f7054 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:30.176517 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbe42e5-eb96-4dbb-b15b-2f39762f7054 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:30.183664 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcdbc54-cb1c-4cf7-a1f6-7b5ec63f5fac None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:30.184887 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 152/278] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Aug 29 22:28:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:30.185759 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbe42e5-eb96-4dbb-b15b-2f39762f7054 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:30.227892 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d59973-5dc3-4901-8add-c3f2623afd21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:30.228863 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d59973-5dc3-4901-8add-c3f2623afd21 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:30.229339 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d59973-5dc3-4901-8add-c3f2623afd21 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:30.230213 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d59973-5dc3-4901-8add-c3f2623afd21 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:30.231496 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d59973-5dc3-4901-8add-c3f2623afd21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:30.233796 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d59973-5dc3-4901-8add-c3f2623afd21 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:30.235959 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 127/279] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:28:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:30.237375 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d59973-5dc3-4901-8add-c3f2623afd21 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:30.299641 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbe42e5-eb96-4dbb-b15b-2f39762f7054 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:30.339229 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b0e15-a235-4ac3-836f-7ab910b09aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:30.339826 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b0e15-a235-4ac3-836f-7ab910b09aff None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:30.340104 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b0e15-a235-4ac3-836f-7ab910b09aff None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:30.340367 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b0e15-a235-4ac3-836f-7ab910b09aff None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:30.341058 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5b0e15-a235-4ac3-836f-7ab910b09aff None admin] RBAC: Adding query filter params (name=reader) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:30.341355 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5b0e15-a235-4ac3-836f-7ab910b09aff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:30.342711 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5b0e15-a235-4ac3-836f-7ab910b09aff None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:30.347488 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d59973-5dc3-4901-8add-c3f2623afd21 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:30.352285 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 153/280] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Aug 29 22:28:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:30.353620 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b0e15-a235-4ac3-836f-7ab910b09aff None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:30.380600 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:30.386110 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd5538f-313c-47ac-99e3-b28ec055b8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:30.386741 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd5538f-313c-47ac-99e3-b28ec055b8fc None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:30.387013 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd5538f-313c-47ac-99e3-b28ec055b8fc None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:30.387279 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd5538f-313c-47ac-99e3-b28ec055b8fc None admin] PATH_INFO: `/v3/users/4c2d26b5deb2401f9be14573048aa671` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:30.402223 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd5538f-313c-47ac-99e3-b28ec055b8fc None admin] RBAC: Authorizing `identity:get_user(user_id=4c2d26b5deb2401f9be14573048aa671)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:30.403709 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd5538f-313c-47ac-99e3-b28ec055b8fc None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:30.410794 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 128/281] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:28:30 2022] GET /identity/v3/users/4c2d26b5deb2401f9be14573048aa671 => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:30.410794 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd5538f-313c-47ac-99e3-b28ec055b8fc None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:30.417182 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:30.485219 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b0e15-a235-4ac3-836f-7ab910b09aff None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:30.523423 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4919d4-f32f-4581-999b-53db67c76d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:30.524028 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4919d4-f32f-4581-999b-53db67c76d3e None admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:30.524312 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4919d4-f32f-4581-999b-53db67c76d3e None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:30.524587 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4919d4-f32f-4581-999b-53db67c76d3e None admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:30.525263 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4919d4-f32f-4581-999b-53db67c76d3e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:30.526709 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4919d4-f32f-4581-999b-53db67c76d3e None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:30.527367 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd5538f-313c-47ac-99e3-b28ec055b8fc None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:30.566168 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187dee1-878d-49d0-97e6-fc1ec2ea6a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:30.566804 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187dee1-878d-49d0-97e6-fc1ec2ea6a31 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:30.567083 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187dee1-878d-49d0-97e6-fc1ec2ea6a31 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:30.567360 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187dee1-878d-49d0-97e6-fc1ec2ea6a31 None admin] PATH_INFO: `/v3/users/22108af3965045308e960dd21fcf1d3f` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:30.568487 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a187dee1-878d-49d0-97e6-fc1ec2ea6a31 None admin] RBAC: Authorizing `identity:get_user(user_id=22108af3965045308e960dd21fcf1d3f)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:30.569980 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a187dee1-878d-49d0-97e6-fc1ec2ea6a31 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:30.571774 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 129/282] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:28:30 2022] GET /identity/v3/users/22108af3965045308e960dd21fcf1d3f => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:30.572323 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187dee1-878d-49d0-97e6-fc1ec2ea6a31 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:30.606545 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 154/283] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Aug 29 22:28:30 2022] POST /identity/v3/users => generated 293 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:28:30.609660 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4919d4-f32f-4581-999b-53db67c76d3e None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:30.666818 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:30.701889 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187dee1-878d-49d0-97e6-fc1ec2ea6a31 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:30.720521 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:30.758655 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4919d4-f32f-4581-999b-53db67c76d3e None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:30.759515 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d77fc-8601-4fb6-aaae-37b65bc47c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:30.760209 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d77fc-8601-4fb6-aaae-37b65bc47c79 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:30.760465 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d77fc-8601-4fb6-aaae-37b65bc47c79 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:30.760695 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d77fc-8601-4fb6-aaae-37b65bc47c79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:30.761548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d77fc-8601-4fb6-aaae-37b65bc47c79 None admin] RBAC: Adding query filter params (role.id=ae06bb653e3b4990a9ddaa1a649c7e06, user.id=4c2d26b5deb2401f9be14573048aa671, scope.system=all) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:30.763392 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d77fc-8601-4fb6-aaae-37b65bc47c79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:30.763564 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d77fc-8601-4fb6-aaae-37b65bc47c79 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:30.772824 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 130/284] 10.222.0.52 () {56 vars in 1308 bytes} [Mon Aug 29 22:28:30 2022] GET /identity/v3/role_assignments?user.id=4c2d26b5deb2401f9be14573048aa671&scope.system=all&role.id=ae06bb653e3b4990a9ddaa1a649c7e06 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:30.804313 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426a775-c1d6-4644-b9c6-b06e9f2d31f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:30.804999 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426a775-c1d6-4644-b9c6-b06e9f2d31f8 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:30.805280 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426a775-c1d6-4644-b9c6-b06e9f2d31f8 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:30.805553 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426a775-c1d6-4644-b9c6-b06e9f2d31f8 None admin] PATH_INFO: `/v3/projects/baba6492015a48bfbca2121065b933ab` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:30.808756 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0426a775-c1d6-4644-b9c6-b06e9f2d31f8 None admin] RBAC: Authorizing `identity:get_project(project_id=baba6492015a48bfbca2121065b933ab)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:30.809652 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0426a775-c1d6-4644-b9c6-b06e9f2d31f8 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:30.811564 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 155/285] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:28:30 2022] GET /identity/v3/projects/baba6492015a48bfbca2121065b933ab => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:30.813787 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ac474-fe96-4018-bbbc-f7c1068de150 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:30.814298 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ac474-fe96-4018-bbbc-f7c1068de150 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:30.815368 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ac474-fe96-4018-bbbc-f7c1068de150 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:30.817350 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 131/286] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:30.818186 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426a775-c1d6-4644-b9c6-b06e9f2d31f8 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:30.825556 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8128ed-a97e-4d1a-aea7-935bb0520831 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:30.825556 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8128ed-a97e-4d1a-aea7-935bb0520831 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:30.825556 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8128ed-a97e-4d1a-aea7-935bb0520831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:30.852857 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-ce8128ed-a97e-4d1a-aea7-935bb0520831 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:30.854647 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-ce8128ed-a97e-4d1a-aea7-935bb0520831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:30.956413 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 132/287] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 22:28:30.972998 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426a775-c1d6-4644-b9c6-b06e9f2d31f8 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:31.010912 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf961d0a-3bb0-46b5-a802-bbd3ef9ce5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:31.011576 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf961d0a-3bb0-46b5-a802-bbd3ef9ce5ba None admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:31.011920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf961d0a-3bb0-46b5-a802-bbd3ef9ce5ba None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:31.012425 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf961d0a-3bb0-46b5-a802-bbd3ef9ce5ba None admin] PATH_INFO: `/v3/projects/baba6492015a48bfbca2121065b933ab/users/22108af3965045308e960dd21fcf1d3f/roles/b9691ffb7caa429ab9a5876dd3dc2ba3` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:31.014287 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf961d0a-3bb0-46b5-a802-bbd3ef9ce5ba None admin] RBAC: Authorizing `identity:create_grant(project_id=baba6492015a48bfbca2121065b933ab, user_id=22108af3965045308e960dd21fcf1d3f, role_id=b9691ffb7caa429ab9a5876dd3dc2ba3)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:31.015744 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf961d0a-3bb0-46b5-a802-bbd3ef9ce5ba None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:31.095348 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:31.099954 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 156/288] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Aug 29 22:28:30 2022] PUT /identity/v3/projects/baba6492015a48bfbca2121065b933ab/users/22108af3965045308e960dd21fcf1d3f/roles/b9691ffb7caa429ab9a5876dd3dc2ba3 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:28:31.277055 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8128ed-a97e-4d1a-aea7-935bb0520831 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:31.368865 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:31.407733 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8128ed-a97e-4d1a-aea7-935bb0520831 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:31.447486 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c049f6e-a822-446f-a7c6-f40333154e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:31.448490 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c049f6e-a822-446f-a7c6-f40333154e6b None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:31.449001 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c049f6e-a822-446f-a7c6-f40333154e6b None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:31.449478 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c049f6e-a822-446f-a7c6-f40333154e6b None admin] PATH_INFO: `/v3/roles/member` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:31.460840 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c049f6e-a822-446f-a7c6-f40333154e6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:31.466018 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-4c049f6e-a822-446f-a7c6-f40333154e6b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:28:31.466018 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:31.466018 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:31.466018 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:31.466018 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:31.466018 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:31.466018 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:31.466018 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:31.466018 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:31.466018 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:31.466018 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:31.466018 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:31.466018 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:31.466018 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:31.466018 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:31.466018 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:31.466877 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:31.466877 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:31.466877 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:31.466877 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:31.466877 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:31.466877 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:31.466877 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:31.466877 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:31.466877 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:31.466877 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:31.466877 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:31.466877 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:31.466877 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:31.466877 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:31.466877 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:31.466877 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:31.467659 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:31.467659 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:31.467659 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:31.467659 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:31.467659 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:31.467659 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:31.467659 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:31.467659 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:31.467659 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:31.467659 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:31.467659 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:28:31.467659 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:28:31.470239 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 133/289] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:28:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:31.476219 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf961d0a-3bb0-46b5-a802-bbd3ef9ce5ba None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:31.654671 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf961d0a-3bb0-46b5-a802-bbd3ef9ce5ba None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:31.707465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3fd061-d7b6-4d67-99c8-3fd276c751f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:31.708481 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3fd061-d7b6-4d67-99c8-3fd276c751f5 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:31.708787 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3fd061-d7b6-4d67-99c8-3fd276c751f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:31.709070 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3fd061-d7b6-4d67-99c8-3fd276c751f5 None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:31.709760 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3fd061-d7b6-4d67-99c8-3fd276c751f5 None admin] RBAC: Adding query filter params (name=member) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:31.710068 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3fd061-d7b6-4d67-99c8-3fd276c751f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:31.711491 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3fd061-d7b6-4d67-99c8-3fd276c751f5 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:31.728389 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 157/290] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Aug 29 22:28:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:31.735931 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c049f6e-a822-446f-a7c6-f40333154e6b None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:31.881170 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c049f6e-a822-446f-a7c6-f40333154e6b None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:31.910705 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:31.917396 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c035a-b119-46f7-b188-2605b8495c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:31.917936 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c035a-b119-46f7-b188-2605b8495c9c None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:31.918476 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c035a-b119-46f7-b188-2605b8495c9c None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:31.918806 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c035a-b119-46f7-b188-2605b8495c9c None admin] PATH_INFO: `/v3/users/2a3ac91cc6fe477aa8539ad27453b248` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:31.920046 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2c035a-b119-46f7-b188-2605b8495c9c None admin] RBAC: Authorizing `identity:get_user(user_id=2a3ac91cc6fe477aa8539ad27453b248)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:31.921605 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2c035a-b119-46f7-b188-2605b8495c9c None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:31.924048 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 134/291] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:28:31 2022] GET /identity/v3/users/2a3ac91cc6fe477aa8539ad27453b248 => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:31.929560 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3fd061-d7b6-4d67-99c8-3fd276c751f5 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:32.008911 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:32.057932 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3fd061-d7b6-4d67-99c8-3fd276c751f5 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:32.106884 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7c4160-2c25-4b7a-b1ee-1068f601b1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:32.107557 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c4160-2c25-4b7a-b1ee-1068f601b1fc None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:32.107839 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c4160-2c25-4b7a-b1ee-1068f601b1fc None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:32.108170 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c4160-2c25-4b7a-b1ee-1068f601b1fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:32.210712 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7c4160-2c25-4b7a-b1ee-1068f601b1fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:32.212255 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7c4160-2c25-4b7a-b1ee-1068f601b1fc None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:32.298146 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:32.349711 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 158/292] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:28:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 22:28:32.355313 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c035a-b119-46f7-b188-2605b8495c9c None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:32.484964 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c035a-b119-46f7-b188-2605b8495c9c None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:32.526503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18453828-f010-4edf-a30b-942deacd3ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:32.527132 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18453828-f010-4edf-a30b-942deacd3ba5 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:32.527412 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18453828-f010-4edf-a30b-942deacd3ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:32.527687 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18453828-f010-4edf-a30b-942deacd3ba5 None admin] PATH_INFO: `/v3/projects/baba6492015a48bfbca2121065b933ab` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:32.528895 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18453828-f010-4edf-a30b-942deacd3ba5 None admin] RBAC: Authorizing `identity:get_project(project_id=baba6492015a48bfbca2121065b933ab)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:32.530358 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18453828-f010-4edf-a30b-942deacd3ba5 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:32.532739 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 135/293] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:28:32 2022] GET /identity/v3/projects/baba6492015a48bfbca2121065b933ab => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:32.539694 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7c4160-2c25-4b7a-b1ee-1068f601b1fc None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:32.703496 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7c4160-2c25-4b7a-b1ee-1068f601b1fc None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:32.743970 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4ac18e-db82-483f-891a-3b7e8269cfaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:32.744489 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ac18e-db82-483f-891a-3b7e8269cfaa None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:32.744791 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ac18e-db82-483f-891a-3b7e8269cfaa None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:32.745064 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ac18e-db82-483f-891a-3b7e8269cfaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:32.745828 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4ac18e-db82-483f-891a-3b7e8269cfaa None admin] RBAC: Adding query filter params (role.id=ae06bb653e3b4990a9ddaa1a649c7e06, scope.project.id=baba6492015a48bfbca2121065b933ab, user.id=2a3ac91cc6fe477aa8539ad27453b248) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:32.746137 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4ac18e-db82-483f-891a-3b7e8269cfaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:32.747551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4ac18e-db82-483f-891a-3b7e8269cfaa None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:32.754493 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 159/294] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:28:32 2022] GET /identity/v3/role_assignments?user.id=2a3ac91cc6fe477aa8539ad27453b248&scope.project.id=baba6492015a48bfbca2121065b933ab&role.id=ae06bb653e3b4990a9ddaa1a649c7e06 => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:28:33.074667 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20bc216-dcd6-48ea-94a0-e7852b2e3d06 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:33.075301 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20bc216-dcd6-48ea-94a0-e7852b2e3d06 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:33.075795 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20bc216-dcd6-48ea-94a0-e7852b2e3d06 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:33.077625 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 136/295] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:33.085807 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7459f4-2d91-4c7f-9a97-c87311663607 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:33.086431 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7459f4-2d91-4c7f-9a97-c87311663607 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:33.086944 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7459f4-2d91-4c7f-9a97-c87311663607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:33.107401 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-2e7459f4-2d91-4c7f-9a97-c87311663607 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:33.109160 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-2e7459f4-2d91-4c7f-9a97-c87311663607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:33.132447 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:33.200902 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 160/296] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:33.298442 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d39ff-1326-455a-9913-0deba313463f None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:33.298790 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d39ff-1326-455a-9913-0deba313463f None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:33.299699 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d39ff-1326-455a-9913-0deba313463f None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:33.301282 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 137/297] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:33.307045 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49db7e6-ddb0-415a-a8bc-0ef3c5c86225 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:33.307366 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49db7e6-ddb0-415a-a8bc-0ef3c5c86225 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:33.307605 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49db7e6-ddb0-415a-a8bc-0ef3c5c86225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:33.326707 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-e49db7e6-ddb0-415a-a8bc-0ef3c5c86225 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:33.327814 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-e49db7e6-ddb0-415a-a8bc-0ef3c5c86225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:33.367814 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc1738-2f77-4398-b2bd-3c71d566dd86 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:33.368310 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc1738-2f77-4398-b2bd-3c71d566dd86 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:33.368415 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc1738-2f77-4398-b2bd-3c71d566dd86 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:33.369961 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 138/298] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:33.383777 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf1942-bef2-44d1-9592-8b7ffedacdef None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:33.384218 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf1942-bef2-44d1-9592-8b7ffedacdef None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:33.384608 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf1942-bef2-44d1-9592-8b7ffedacdef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:33.405972 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-01bf1942-bef2-44d1-9592-8b7ffedacdef None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:33.407365 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-01bf1942-bef2-44d1-9592-8b7ffedacdef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:33.408908 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 161/299] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 22:28:33.476906 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:33.485086 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 139/300] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 22:28:33.506996 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49db7e6-ddb0-415a-a8bc-0ef3c5c86225 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:33.600461 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:33.616844 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49db7e6-ddb0-415a-a8bc-0ef3c5c86225 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:33.653160 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4072d9-7653-4966-ac76-f37c0472740f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:33.653793 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4072d9-7653-4966-ac76-f37c0472740f None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:33.654160 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4072d9-7653-4966-ac76-f37c0472740f None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:33.654440 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4072d9-7653-4966-ac76-f37c0472740f None admin] PATH_INFO: `/v3/domains/default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:33.655565 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4072d9-7653-4966-ac76-f37c0472740f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:33.657045 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4072d9-7653-4966-ac76-f37c0472740f None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:33.659710 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 162/301] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:28:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:33.665050 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf1942-bef2-44d1-9592-8b7ffedacdef None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:33.716152 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4072d9-7653-4966-ac76-f37c0472740f None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:33.780628 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf1942-bef2-44d1-9592-8b7ffedacdef None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:33.818995 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b39751-f24d-4cca-b524-8484e5f0ef2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:33.819600 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b39751-f24d-4cca-b524-8484e5f0ef2e None admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:33.819882 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b39751-f24d-4cca-b524-8484e5f0ef2e None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:33.820154 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b39751-f24d-4cca-b524-8484e5f0ef2e None admin] PATH_INFO: `/v3/groups` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:33.820819 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b39751-f24d-4cca-b524-8484e5f0ef2e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:33.822286 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b39751-f24d-4cca-b524-8484e5f0ef2e None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:33.833604 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4072d9-7653-4966-ac76-f37c0472740f None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:33.867090 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394a3cc2-4fef-4dda-9599-2383d481929d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:33.867630 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a3cc2-4fef-4dda-9599-2383d481929d None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:33.867908 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a3cc2-4fef-4dda-9599-2383d481929d None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:33.868181 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a3cc2-4fef-4dda-9599-2383d481929d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:33.878113 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394a3cc2-4fef-4dda-9599-2383d481929d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:33.882969 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-394a3cc2-4fef-4dda-9599-2383d481929d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 22:28:33.882969 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:33.882969 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:33.882969 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:33.882969 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:33.882969 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:33.882969 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:33.882969 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:33.882969 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:33.882969 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:33.882969 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:33.882969 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:33.882969 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:33.882969 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:33.882969 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:33.882969 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:33.883982 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:33.883982 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:33.883982 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:33.883982 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:33.883982 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:33.883982 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:33.883982 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:33.883982 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:33.883982 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:33.883982 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:33.883982 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:33.883982 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:33.883982 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:33.883982 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:33.883982 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:33.883982 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:33.884829 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:33.884829 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:33.884829 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:33.884829 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:33.884829 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:33.884829 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:33.884829 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:33.884829 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:33.884829 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:33.884829 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:33.884829 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 22:28:33.884829 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:28:33.884829 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 163/302] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:28:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:33.884829 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394a3cc2-4fef-4dda-9599-2383d481929d None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:33.893685 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 140/303] 10.222.0.52 () {60 vars in 1134 bytes} [Mon Aug 29 22:28:33 2022] POST /identity/v3/groups => generated 229 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:28:33.895110 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6132d81-9ba3-4710-b135-783e1abc94a0 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:33.895542 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6132d81-9ba3-4710-b135-783e1abc94a0 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:33.895819 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6132d81-9ba3-4710-b135-783e1abc94a0 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:33.897090 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 141/304] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:33.902103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6132d81-9ba3-4710-b135-783e1abc94a0 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:33.915258 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:33.973800 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:34.023050 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394a3cc2-4fef-4dda-9599-2383d481929d None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:34.046979 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6132d81-9ba3-4710-b135-783e1abc94a0 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:34.069270 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e227c7-912f-430a-bc4d-a8f4e680d582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:34.070242 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e227c7-912f-430a-bc4d-a8f4e680d582 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:34.070773 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e227c7-912f-430a-bc4d-a8f4e680d582 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:34.071248 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e227c7-912f-430a-bc4d-a8f4e680d582 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:34.088391 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e227c7-912f-430a-bc4d-a8f4e680d582 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:34.093343 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-e7e227c7-912f-430a-bc4d-a8f4e680d582 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:28:34.093343 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:34.093343 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:34.093343 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:34.093343 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:34.093343 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:34.093343 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:34.093343 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:34.093343 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:34.093343 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:34.093343 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:34.093343 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:34.093343 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:34.093343 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:34.093343 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:34.093343 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:34.094222 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:34.094222 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:34.094222 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:34.094222 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:34.094222 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:34.094222 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:34.094222 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:34.094222 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:34.094222 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:34.094222 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:34.094222 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:34.094222 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:34.094222 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:34.094222 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:34.094222 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:34.094222 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:34.095000 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:34.095000 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:34.095000 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:34.095000 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:34.095000 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:34.095000 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:34.095000 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:34.095000 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:34.095000 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:34.095000 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:34.095000 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:28:34.095000 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:28:34.095000 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 164/305] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:28:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:34.096876 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ddce2c-ae88-4eb9-af7f-de58d9833cf9 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:34.101049 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ddce2c-ae88-4eb9-af7f-de58d9833cf9 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:34.101364 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ddce2c-ae88-4eb9-af7f-de58d9833cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:34.112349 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c428e08-71eb-49ba-8c60-8242dd4870b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:34.113467 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c428e08-71eb-49ba-8c60-8242dd4870b3 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:34.113970 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c428e08-71eb-49ba-8c60-8242dd4870b3 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:34.114440 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c428e08-71eb-49ba-8c60-8242dd4870b3 None admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:34.116264 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c428e08-71eb-49ba-8c60-8242dd4870b3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:34.116857 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c428e08-71eb-49ba-8c60-8242dd4870b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:34.118521 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c428e08-71eb-49ba-8c60-8242dd4870b3 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:34.129000 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-b8ddce2c-ae88-4eb9-af7f-de58d9833cf9 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:34.130210 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-b8ddce2c-ae88-4eb9-af7f-de58d9833cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:34.131068 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 142/306] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Aug 29 22:28:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:34.136329 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c428e08-71eb-49ba-8c60-8242dd4870b3 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:34.204756 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 165/307] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:34.206367 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ddce2c-ae88-4eb9-af7f-de58d9833cf9 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:34.209514 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:34.304044 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c428e08-71eb-49ba-8c60-8242dd4870b3 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:34.357015 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:34.358076 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ddce2c-ae88-4eb9-af7f-de58d9833cf9 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:34.397338 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03006d7a-ee2c-4746-8e05-e74ed2b0af12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:34.397903 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03006d7a-ee2c-4746-8e05-e74ed2b0af12 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:34.398833 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03006d7a-ee2c-4746-8e05-e74ed2b0af12 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:34.398941 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03006d7a-ee2c-4746-8e05-e74ed2b0af12 None admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:34.399605 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03006d7a-ee2c-4746-8e05-e74ed2b0af12 None admin] RBAC: Adding query filter params (name=member) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:34.399916 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03006d7a-ee2c-4746-8e05-e74ed2b0af12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:34.402354 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03006d7a-ee2c-4746-8e05-e74ed2b0af12 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:34.414758 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 143/308] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Aug 29 22:28:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:34.419597 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03006d7a-ee2c-4746-8e05-e74ed2b0af12 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:34.428652 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e877b49-5f25-41bf-ab69-e73beac4672b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:34.430268 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e877b49-5f25-41bf-ab69-e73beac4672b None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:34.434776 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e877b49-5f25-41bf-ab69-e73beac4672b None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:34.435165 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e877b49-5f25-41bf-ab69-e73beac4672b None admin] PATH_INFO: `/v3/users/2100ee3c17404419be6a50b9a6cef195` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:34.459507 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e877b49-5f25-41bf-ab69-e73beac4672b None admin] RBAC: Authorizing `identity:get_user(user_id=2100ee3c17404419be6a50b9a6cef195)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:34.462251 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e877b49-5f25-41bf-ab69-e73beac4672b None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:34.464899 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 166/309] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:28:34 2022] GET /identity/v3/users/2100ee3c17404419be6a50b9a6cef195 => generated 293 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:34.473865 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e877b49-5f25-41bf-ab69-e73beac4672b None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:34.546853 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03006d7a-ee2c-4746-8e05-e74ed2b0af12 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:34.568861 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:34.586506 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5a5616-bc46-4869-93d0-b4cb74284523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:34.587138 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a5616-bc46-4869-93d0-b4cb74284523 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:34.587420 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a5616-bc46-4869-93d0-b4cb74284523 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:34.587697 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a5616-bc46-4869-93d0-b4cb74284523 None admin] PATH_INFO: `/v3/users/4c2d26b5deb2401f9be14573048aa671` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:34.588867 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5a5616-bc46-4869-93d0-b4cb74284523 None admin] RBAC: Authorizing `identity:get_user(user_id=4c2d26b5deb2401f9be14573048aa671)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:34.590330 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5a5616-bc46-4869-93d0-b4cb74284523 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:34.596888 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 144/310] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:28:34 2022] GET /identity/v3/users/4c2d26b5deb2401f9be14573048aa671 => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:28:34.596888 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5a5616-bc46-4869-93d0-b4cb74284523 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:34.615175 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e877b49-5f25-41bf-ab69-e73beac4672b None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:34.648276 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:34.654429 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c643ca8d-cb0c-42ed-841c-fe386fbf12b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:34.655091 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643ca8d-cb0c-42ed-841c-fe386fbf12b6 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:34.655372 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643ca8d-cb0c-42ed-841c-fe386fbf12b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:34.655647 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643ca8d-cb0c-42ed-841c-fe386fbf12b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:34.656362 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c643ca8d-cb0c-42ed-841c-fe386fbf12b6 None admin] RBAC: Adding query filter params (role.id=b9691ffb7caa429ab9a5876dd3dc2ba3, user.id=2100ee3c17404419be6a50b9a6cef195, scope.system=all) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:34.657572 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c643ca8d-cb0c-42ed-841c-fe386fbf12b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:34.658114 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c643ca8d-cb0c-42ed-841c-fe386fbf12b6 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:34.664210 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 167/311] 10.222.0.52 () {56 vars in 1308 bytes} [Mon Aug 29 22:28:34 2022] GET /identity/v3/role_assignments?user.id=2100ee3c17404419be6a50b9a6cef195&scope.system=all&role.id=b9691ffb7caa429ab9a5876dd3dc2ba3 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:34.664816 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c643ca8d-cb0c-42ed-841c-fe386fbf12b6 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:34.711591 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5a5616-bc46-4869-93d0-b4cb74284523 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:34.751782 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ad07c-e805-43b8-9f99-6bbcc856e32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:34.753401 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ad07c-e805-43b8-9f99-6bbcc856e32d None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:34.753878 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ad07c-e805-43b8-9f99-6bbcc856e32d None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:34.754344 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ad07c-e805-43b8-9f99-6bbcc856e32d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:34.767143 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797ad07c-e805-43b8-9f99-6bbcc856e32d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:34.773842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-797ad07c-e805-43b8-9f99-6bbcc856e32d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 22:28:34.773842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:34.773842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:34.773842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:34.773842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:34.773842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:34.773842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:34.773842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:34.773842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:34.773842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:34.773842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:34.773842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:34.773842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:34.773842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:34.773842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:34.773842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:34.774782 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:34.774782 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:34.774782 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:34.774782 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:34.774782 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:34.774782 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:34.774782 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:34.774782 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:34.774782 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:34.774782 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:34.774782 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:34.774782 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:34.774782 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:34.774782 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:34.774782 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:34.774782 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:34.775572 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:34.775572 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:34.775572 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:34.775572 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:34.775572 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:34.775572 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:34.775572 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:34.775572 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:34.775572 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:34.775572 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:34.775572 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 22:28:34.775572 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:28:34.778181 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 145/312] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:28:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:34.784910 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ad07c-e805-43b8-9f99-6bbcc856e32d None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:34.795343 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c643ca8d-cb0c-42ed-841c-fe386fbf12b6 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:34.833393 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:34.845955 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c05c4c-b558-4cb8-bcf5-188624345eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:34.847629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c05c4c-b558-4cb8-bcf5-188624345eed None admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:34.848095 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c05c4c-b558-4cb8-bcf5-188624345eed None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:34.848861 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c05c4c-b558-4cb8-bcf5-188624345eed None admin] PATH_INFO: `/v3/system/users/4c2d26b5deb2401f9be14573048aa671/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:34.853202 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c05c4c-b558-4cb8-bcf5-188624345eed None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=4c2d26b5deb2401f9be14573048aa671, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:34.853202 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c05c4c-b558-4cb8-bcf5-188624345eed None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:34.898865 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ad07c-e805-43b8-9f99-6bbcc856e32d None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:34.909088 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:34.936125 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc8b806-c2cf-44b2-a854-8f0349df423f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:34.938757 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc8b806-c2cf-44b2-a854-8f0349df423f None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:34.938757 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc8b806-c2cf-44b2-a854-8f0349df423f None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:34.938757 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc8b806-c2cf-44b2-a854-8f0349df423f None admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:34.938757 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc8b806-c2cf-44b2-a854-8f0349df423f None admin] RBAC: Adding query filter params (name=reader) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:34.938757 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc8b806-c2cf-44b2-a854-8f0349df423f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:34.940106 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc8b806-c2cf-44b2-a854-8f0349df423f None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:34.949370 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 146/313] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Aug 29 22:28:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:34.954929 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc8b806-c2cf-44b2-a854-8f0349df423f None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:34.969040 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 168/314] 10.222.0.52 () {58 vars in 1334 bytes} [Mon Aug 29 22:28:34 2022] PUT /identity/v3/system/users/4c2d26b5deb2401f9be14573048aa671/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:28:35.112844 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc8b806-c2cf-44b2-a854-8f0349df423f None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:35.162650 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede96be-dca3-49f9-92a1-3a8d9ae6748d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:35.163225 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede96be-dca3-49f9-92a1-3a8d9ae6748d None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:35.163506 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede96be-dca3-49f9-92a1-3a8d9ae6748d None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:35.163782 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede96be-dca3-49f9-92a1-3a8d9ae6748d None admin] PATH_INFO: `/v3/users/22108af3965045308e960dd21fcf1d3f` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:35.165281 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ede96be-dca3-49f9-92a1-3a8d9ae6748d None admin] RBAC: Authorizing `identity:get_user(user_id=22108af3965045308e960dd21fcf1d3f)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:35.167753 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ede96be-dca3-49f9-92a1-3a8d9ae6748d None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:35.170252 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 147/315] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:28:34 2022] GET /identity/v3/users/22108af3965045308e960dd21fcf1d3f => generated 297 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:28:35.179433 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c05c4c-b558-4cb8-bcf5-188624345eed None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:35.309147 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c05c4c-b558-4cb8-bcf5-188624345eed None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:35.356039 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea598cc2-259f-475c-a11e-b0115146a3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:35.356653 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea598cc2-259f-475c-a11e-b0115146a3f7 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:35.356966 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea598cc2-259f-475c-a11e-b0115146a3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:35.357563 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea598cc2-259f-475c-a11e-b0115146a3f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:35.434496 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:35.447435 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea598cc2-259f-475c-a11e-b0115146a3f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:35.454896 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea598cc2-259f-475c-a11e-b0115146a3f7 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:35.580861 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 169/316] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:28:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:28:35.587152 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede96be-dca3-49f9-92a1-3a8d9ae6748d None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:35.646027 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:35.720476 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede96be-dca3-49f9-92a1-3a8d9ae6748d None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:35.761100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9ba35e-07e8-4569-93e9-4e6c4b9420ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:35.761738 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ba35e-07e8-4569-93e9-4e6c4b9420ae None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:35.762020 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ba35e-07e8-4569-93e9-4e6c4b9420ae None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:35.762296 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ba35e-07e8-4569-93e9-4e6c4b9420ae None admin] PATH_INFO: `/v3/projects/baba6492015a48bfbca2121065b933ab` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:35.763450 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9ba35e-07e8-4569-93e9-4e6c4b9420ae None admin] RBAC: Authorizing `identity:get_project(project_id=baba6492015a48bfbca2121065b933ab)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:35.765831 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9ba35e-07e8-4569-93e9-4e6c4b9420ae None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:35.768402 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 148/317] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:28:35 2022] GET /identity/v3/projects/baba6492015a48bfbca2121065b933ab => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:35.776584 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea598cc2-259f-475c-a11e-b0115146a3f7 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:35.905534 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:35.916629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea598cc2-259f-475c-a11e-b0115146a3f7 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:35.962120 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b945d2d6-0922-4fa5-924c-abe143b39d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:35.963289 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b945d2d6-0922-4fa5-924c-abe143b39d92 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:35.964159 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b945d2d6-0922-4fa5-924c-abe143b39d92 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:35.964440 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b945d2d6-0922-4fa5-924c-abe143b39d92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:35.965212 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b945d2d6-0922-4fa5-924c-abe143b39d92 None admin] RBAC: Adding query filter params (role.id=b9691ffb7caa429ab9a5876dd3dc2ba3, scope.project.id=baba6492015a48bfbca2121065b933ab, user.id=22108af3965045308e960dd21fcf1d3f) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:35.965526 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b945d2d6-0922-4fa5-924c-abe143b39d92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:35.967069 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b945d2d6-0922-4fa5-924c-abe143b39d92 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:35.977080 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 170/318] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:28:35 2022] GET /identity/v3/role_assignments?user.id=22108af3965045308e960dd21fcf1d3f&scope.project.id=baba6492015a48bfbca2121065b933ab&role.id=b9691ffb7caa429ab9a5876dd3dc2ba3 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:36.126462 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19686267-6810-465e-8d4b-53ca7af3c2ee None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:36.127278 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19686267-6810-465e-8d4b-53ca7af3c2ee None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:36.127821 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19686267-6810-465e-8d4b-53ca7af3c2ee None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:36.130057 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 149/319] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:36.137969 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726d820-79dd-4149-8fd8-7d8a99359b40 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:36.138272 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726d820-79dd-4149-8fd8-7d8a99359b40 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:36.138530 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726d820-79dd-4149-8fd8-7d8a99359b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:36.160501 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-4726d820-79dd-4149-8fd8-7d8a99359b40 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:36.162261 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-4726d820-79dd-4149-8fd8-7d8a99359b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:36.245640 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 171/320] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:36.517043 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19686267-6810-465e-8d4b-53ca7af3c2ee None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:36.632801 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19686267-6810-465e-8d4b-53ca7af3c2ee None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:36.640670 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:36.669589 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4844649-ffaa-4a0b-a8c8-65f8a2f80afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:36.670220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4844649-ffaa-4a0b-a8c8-65f8a2f80afc None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:36.670497 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4844649-ffaa-4a0b-a8c8-65f8a2f80afc None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:36.670770 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4844649-ffaa-4a0b-a8c8-65f8a2f80afc None admin] PATH_INFO: `/v3/domains/default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:36.671875 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4844649-ffaa-4a0b-a8c8-65f8a2f80afc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:36.673618 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4844649-ffaa-4a0b-a8c8-65f8a2f80afc None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:36.675954 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 150/321] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:28:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:36.681253 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4726d820-79dd-4149-8fd8-7d8a99359b40 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:36.756734 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5d606-f2da-4580-832a-12861b9777f3 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:36.757094 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5d606-f2da-4580-832a-12861b9777f3 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:36.757318 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5d606-f2da-4580-832a-12861b9777f3 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:36.758806 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 151/322] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:28:36.763497 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ec7b4-8911-4f28-8b09-7cfddd1d21fb None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:36.763779 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ec7b4-8911-4f28-8b09-7cfddd1d21fb None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:36.764059 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ec7b4-8911-4f28-8b09-7cfddd1d21fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:36.768321 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:36.784621 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-b95ec7b4-8911-4f28-8b09-7cfddd1d21fb None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:36.785835 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-b95ec7b4-8911-4f28-8b09-7cfddd1d21fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:36.786011 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4726d820-79dd-4149-8fd8-7d8a99359b40 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:36.825063 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90818352-8efc-42b2-a282-e77fa883bc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:36.825678 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90818352-8efc-42b2-a282-e77fa883bc83 None admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:36.825956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90818352-8efc-42b2-a282-e77fa883bc83 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:36.826227 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90818352-8efc-42b2-a282-e77fa883bc83 None admin] PATH_INFO: `/v3/groups` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:36.826864 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90818352-8efc-42b2-a282-e77fa883bc83 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:36.828294 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90818352-8efc-42b2-a282-e77fa883bc83 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:36.857101 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 152/323] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 22:28:36.927005 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 172/324] 10.222.0.52 () {60 vars in 1134 bytes} [Mon Aug 29 22:28:36 2022] POST /identity/v3/groups => generated 226 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:28:37.034726 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51ee45-5140-42da-baf1-a20aab91be23 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:37.034901 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51ee45-5140-42da-baf1-a20aab91be23 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:37.035180 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51ee45-5140-42da-baf1-a20aab91be23 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:37.036767 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 153/325] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:37.042389 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6c21e-7500-4ce3-9e34-05c5b4b1eb3a None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:37.042669 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6c21e-7500-4ce3-9e34-05c5b4b1eb3a None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:37.042955 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6c21e-7500-4ce3-9e34-05c5b4b1eb3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:37.063096 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-c4e6c21e-7500-4ce3-9e34-05c5b4b1eb3a None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:37.064292 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-c4e6c21e-7500-4ce3-9e34-05c5b4b1eb3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:37.136704 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc51ee45-5140-42da-baf1-a20aab91be23 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:37.209390 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 173/326] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:37.256983 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc51ee45-5140-42da-baf1-a20aab91be23 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:37.260565 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:37.296134 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82ede03-4d17-4a7a-a63d-37a03c9ac51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:37.297080 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ede03-4d17-4a7a-a63d-37a03c9ac51b None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:37.297581 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ede03-4d17-4a7a-a63d-37a03c9ac51b None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:37.298496 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ede03-4d17-4a7a-a63d-37a03c9ac51b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:37.309445 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82ede03-4d17-4a7a-a63d-37a03c9ac51b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:37.315260 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-e82ede03-4d17-4a7a-a63d-37a03c9ac51b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 22:28:37.315260 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:37.315260 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:37.315260 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:37.315260 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:37.315260 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:37.315260 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:37.315260 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:37.315260 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:37.315260 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:37.315260 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:37.315260 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:37.315260 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:37.315260 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:37.315260 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:37.315260 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:37.316322 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:37.316322 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:37.316322 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:37.316322 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:37.316322 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:37.316322 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:37.316322 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:37.316322 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:37.316322 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:37.316322 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:37.316322 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:37.316322 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:37.316322 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:37.316322 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:37.316322 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:37.316322 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:37.317213 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:37.317213 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:37.317213 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:37.317213 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:37.317213 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:37.317213 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:37.317213 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:37.317213 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:37.317213 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:37.317213 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:37.317213 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 22:28:37.317213 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:28:37.318376 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 154/327] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:28:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:28:37.324243 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6c21e-7500-4ce3-9e34-05c5b4b1eb3a None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:37.416507 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:37.447876 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6c21e-7500-4ce3-9e34-05c5b4b1eb3a None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:37.491299 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90743796-f111-47c9-875e-1d2687ff9f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:37.492007 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90743796-f111-47c9-875e-1d2687ff9f14 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:37.492293 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90743796-f111-47c9-875e-1d2687ff9f14 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:37.492569 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90743796-f111-47c9-875e-1d2687ff9f14 None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:37.493323 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90743796-f111-47c9-875e-1d2687ff9f14 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:37.493632 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90743796-f111-47c9-875e-1d2687ff9f14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:37.495016 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90743796-f111-47c9-875e-1d2687ff9f14 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:37.506231 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82ede03-4d17-4a7a-a63d-37a03c9ac51b None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:37.512684 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 174/328] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Aug 29 22:28:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:37.518935 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90743796-f111-47c9-875e-1d2687ff9f14 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:37.645428 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90743796-f111-47c9-875e-1d2687ff9f14 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:37.665703 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82ede03-4d17-4a7a-a63d-37a03c9ac51b None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:37.686892 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a8ef38-ff81-459e-a8a9-70b547d41c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:37.687922 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8ef38-ff81-459e-a8a9-70b547d41c39 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:37.688319 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8ef38-ff81-459e-a8a9-70b547d41c39 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:37.688602 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8ef38-ff81-459e-a8a9-70b547d41c39 None admin] PATH_INFO: `/v3/users/2100ee3c17404419be6a50b9a6cef195` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:37.690410 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a8ef38-ff81-459e-a8a9-70b547d41c39 None admin] RBAC: Authorizing `identity:get_user(user_id=2100ee3c17404419be6a50b9a6cef195)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:37.692884 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a8ef38-ff81-459e-a8a9-70b547d41c39 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:37.695597 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 175/329] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:28:37 2022] GET /identity/v3/users/2100ee3c17404419be6a50b9a6cef195 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:28:37.702426 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a8ef38-ff81-459e-a8a9-70b547d41c39 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:37.709982 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:37.712365 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eea353-934a-4c2f-8e0a-f73653340d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:37.712613 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eea353-934a-4c2f-8e0a-f73653340d6b None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:37.712959 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eea353-934a-4c2f-8e0a-f73653340d6b None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:37.713236 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eea353-934a-4c2f-8e0a-f73653340d6b None admin] PATH_INFO: `/v3/roles/member` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:37.730192 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eea353-934a-4c2f-8e0a-f73653340d6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:37.736464 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-d2eea353-934a-4c2f-8e0a-f73653340d6b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:28:37.736464 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:37.736464 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:37.736464 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:37.736464 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:37.736464 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:37.736464 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:37.736464 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:37.736464 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:37.736464 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:37.736464 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:37.736464 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:37.736464 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:37.736464 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:37.736464 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:37.736464 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:37.737888 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:37.737888 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:37.737888 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:37.737888 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:37.737888 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:37.737888 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:37.737888 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:37.737888 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:37.737888 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:37.737888 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:37.737888 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:37.737888 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:37.737888 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:37.737888 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:37.737888 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:37.737888 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:37.738681 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:37.738681 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:37.738681 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:37.738681 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:37.738681 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:37.738681 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:37.738681 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:37.738681 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:37.738681 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:37.738681 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:37.738681 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:28:37.738681 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:28:37.740943 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 155/330] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:28:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:28:37.746119 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eea353-934a-4c2f-8e0a-f73653340d6b None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:37.777491 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:37.848115 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a8ef38-ff81-459e-a8a9-70b547d41c39 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:37.874407 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eea353-934a-4c2f-8e0a-f73653340d6b None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:37.892189 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd36a898-cfd8-4b53-8e5a-b60249bbe8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:37.894378 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd36a898-cfd8-4b53-8e5a-b60249bbe8e5 None admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:37.895196 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd36a898-cfd8-4b53-8e5a-b60249bbe8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:37.896042 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd36a898-cfd8-4b53-8e5a-b60249bbe8e5 None admin] PATH_INFO: `/v3/system/users/2100ee3c17404419be6a50b9a6cef195/roles/b9691ffb7caa429ab9a5876dd3dc2ba3` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:37.897964 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd36a898-cfd8-4b53-8e5a-b60249bbe8e5 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2100ee3c17404419be6a50b9a6cef195, role_id=b9691ffb7caa429ab9a5876dd3dc2ba3)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:37.899635 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd36a898-cfd8-4b53-8e5a-b60249bbe8e5 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:37.913962 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f2735-cb53-4cfe-89b7-b4a5cc2c15c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:37.914571 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f2735-cb53-4cfe-89b7-b4a5cc2c15c8 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:37.914915 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f2735-cb53-4cfe-89b7-b4a5cc2c15c8 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:37.915191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f2735-cb53-4cfe-89b7-b4a5cc2c15c8 None admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:37.915917 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324f2735-cb53-4cfe-89b7-b4a5cc2c15c8 None admin] RBAC: Adding query filter params (name=member) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:37.916222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324f2735-cb53-4cfe-89b7-b4a5cc2c15c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:37.917628 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324f2735-cb53-4cfe-89b7-b4a5cc2c15c8 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:37.927383 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 156/331] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Aug 29 22:28:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:37.932864 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f2735-cb53-4cfe-89b7-b4a5cc2c15c8 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:37.980272 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 176/332] 10.222.0.52 () {58 vars in 1334 bytes} [Mon Aug 29 22:28:37 2022] PUT /identity/v3/system/users/2100ee3c17404419be6a50b9a6cef195/roles/b9691ffb7caa429ab9a5876dd3dc2ba3 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:28:38.037698 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:38.054834 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f2735-cb53-4cfe-89b7-b4a5cc2c15c8 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:38.093707 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31257067-61cb-4843-8477-097b74a869aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:38.094361 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31257067-61cb-4843-8477-097b74a869aa None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:38.094644 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31257067-61cb-4843-8477-097b74a869aa None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:38.094921 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31257067-61cb-4843-8477-097b74a869aa None admin] PATH_INFO: `/v3/users/4c2d26b5deb2401f9be14573048aa671` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:38.096170 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31257067-61cb-4843-8477-097b74a869aa None admin] RBAC: Authorizing `identity:get_user(user_id=4c2d26b5deb2401f9be14573048aa671)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:38.097694 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31257067-61cb-4843-8477-097b74a869aa None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:38.099853 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 157/333] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:28:37 2022] GET /identity/v3/users/4c2d26b5deb2401f9be14573048aa671 => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:38.105648 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd36a898-cfd8-4b53-8e5a-b60249bbe8e5 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:38.240409 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:38.279269 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd36a898-cfd8-4b53-8e5a-b60249bbe8e5 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:38.327465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d507b07-8b09-4d75-aa22-2b7b2a78f59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:38.328270 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d507b07-8b09-4d75-aa22-2b7b2a78f59f None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:38.328547 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d507b07-8b09-4d75-aa22-2b7b2a78f59f None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:38.328851 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d507b07-8b09-4d75-aa22-2b7b2a78f59f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:38.329587 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d507b07-8b09-4d75-aa22-2b7b2a78f59f None admin] RBAC: Adding query filter params (role.id=ae06bb653e3b4990a9ddaa1a649c7e06, user.id=4c2d26b5deb2401f9be14573048aa671, scope.system=all) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:38.329895 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d507b07-8b09-4d75-aa22-2b7b2a78f59f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:38.332069 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d507b07-8b09-4d75-aa22-2b7b2a78f59f None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:38.351962 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 177/334] 10.222.0.52 () {56 vars in 1308 bytes} [Mon Aug 29 22:28:38 2022] GET /identity/v3/role_assignments?user.id=4c2d26b5deb2401f9be14573048aa671&scope.system=all&role.id=ae06bb653e3b4990a9ddaa1a649c7e06 => generated 510 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:39.184378 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b600b6-4aec-4015-9dcd-b56005cca467 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:39.185433 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b600b6-4aec-4015-9dcd-b56005cca467 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:39.185918 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b600b6-4aec-4015-9dcd-b56005cca467 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:39.187729 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 158/335] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:39.193560 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3aced6-72e0-4012-b556-3f28d17cf943 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:39.194461 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3aced6-72e0-4012-b556-3f28d17cf943 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:39.194944 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3aced6-72e0-4012-b556-3f28d17cf943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:39.218267 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-ea3aced6-72e0-4012-b556-3f28d17cf943 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:39.219843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-ea3aced6-72e0-4012-b556-3f28d17cf943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:39.303288 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 178/336] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:39.578101 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f936825-17dc-4c9a-8bd0-7d2fbb6cd3cd None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:39.578424 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f936825-17dc-4c9a-8bd0-7d2fbb6cd3cd None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:39.578653 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f936825-17dc-4c9a-8bd0-7d2fbb6cd3cd None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:39.580410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 159/337] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:39.585465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e6fb50-9617-4f62-8aab-227a5664e873 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:39.585753 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e6fb50-9617-4f62-8aab-227a5664e873 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:39.586033 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e6fb50-9617-4f62-8aab-227a5664e873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:39.609015 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-f5e6fb50-9617-4f62-8aab-227a5664e873 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:39.610200 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-f5e6fb50-9617-4f62-8aab-227a5664e873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:39.680273 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:39.697675 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 179/338] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:39.854276 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f936825-17dc-4c9a-8bd0-7d2fbb6cd3cd None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:39.994462 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f936825-17dc-4c9a-8bd0-7d2fbb6cd3cd None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:40.049878 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e6fb50-9617-4f62-8aab-227a5664e873 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:40.100834 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde0bb1c-94f8-4b97-89f5-fd2c9cb61db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:40.101825 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0bb1c-94f8-4b97-89f5-fd2c9cb61db0 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:40.102329 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0bb1c-94f8-4b97-89f5-fd2c9cb61db0 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:40.102852 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0bb1c-94f8-4b97-89f5-fd2c9cb61db0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:40.118654 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde0bb1c-94f8-4b97-89f5-fd2c9cb61db0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:40.125100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-dde0bb1c-94f8-4b97-89f5-fd2c9cb61db0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:28:40.125100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:40.125100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:40.125100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:40.125100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:40.125100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:40.125100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:40.125100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:40.125100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:40.125100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:40.125100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:40.125100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:40.125100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:40.125100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:40.125100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:40.125100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:40.126000 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:40.126000 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:40.126000 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:40.126000 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:40.126000 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:40.126000 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:40.126000 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:40.126000 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:40.126000 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:40.126000 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:40.126000 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:40.126000 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:40.126000 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:40.126000 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:40.126000 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:40.126000 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:40.126765 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:40.126765 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:40.126765 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:40.126765 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:40.126765 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:40.126765 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:40.126765 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:40.126765 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:40.126765 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:40.126765 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:40.126765 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:28:40.126765 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:28:40.129549 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 160/339] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:28:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:40.141272 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde0bb1c-94f8-4b97-89f5-fd2c9cb61db0 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:40.165719 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:40.275697 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e6fb50-9617-4f62-8aab-227a5664e873 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:40.306640 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde0bb1c-94f8-4b97-89f5-fd2c9cb61db0 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:40.317339 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61dac75-35fc-47d2-9d88-325fcefc5b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:40.317981 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61dac75-35fc-47d2-9d88-325fcefc5b08 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:40.318265 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61dac75-35fc-47d2-9d88-325fcefc5b08 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:40.318541 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61dac75-35fc-47d2-9d88-325fcefc5b08 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:40.332202 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61dac75-35fc-47d2-9d88-325fcefc5b08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:40.337165 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-b61dac75-35fc-47d2-9d88-325fcefc5b08 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:28:40.337165 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:40.337165 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:40.337165 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:40.337165 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:40.337165 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:40.337165 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:40.337165 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:40.337165 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:40.337165 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:40.337165 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:40.337165 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:40.337165 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:40.337165 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:40.337165 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:40.337165 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:40.338012 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:40.338012 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:40.338012 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:40.338012 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:40.338012 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:40.338012 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:40.338012 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:40.338012 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:40.338012 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:40.338012 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:40.338012 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:40.338012 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:40.338012 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:40.338012 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:40.338012 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:40.338012 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:40.338820 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:40.338820 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:40.338820 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:40.338820 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:40.338820 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:40.338820 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:40.338820 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:40.338820 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:40.338820 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:40.338820 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:40.338820 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:28:40.338820 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:28:40.338820 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 180/340] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Aug 29 22:28:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:40.349829 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61dac75-35fc-47d2-9d88-325fcefc5b08 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:40.391115 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd5a3e-8c3a-4762-9c74-7e303bae1945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:40.392076 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd5a3e-8c3a-4762-9c74-7e303bae1945 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:40.392683 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd5a3e-8c3a-4762-9c74-7e303bae1945 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:40.393677 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd5a3e-8c3a-4762-9c74-7e303bae1945 None admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:40.394553 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defd5a3e-8c3a-4762-9c74-7e303bae1945 None admin] RBAC: Adding query filter params (name=member) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:40.395084 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defd5a3e-8c3a-4762-9c74-7e303bae1945 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:40.396701 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defd5a3e-8c3a-4762-9c74-7e303bae1945 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:40.419229 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 161/341] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Aug 29 22:28:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:28:40.425171 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:40.453778 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd5a3e-8c3a-4762-9c74-7e303bae1945 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:40.521748 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61dac75-35fc-47d2-9d88-325fcefc5b08 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:40.584183 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395847c-f9ac-48c7-b840-e4ed41972d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:40.584183 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395847c-f9ac-48c7-b840-e4ed41972d38 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:40.584183 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395847c-f9ac-48c7-b840-e4ed41972d38 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:40.584183 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395847c-f9ac-48c7-b840-e4ed41972d38 None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:40.584580 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7395847c-f9ac-48c7-b840-e4ed41972d38 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:40.584580 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7395847c-f9ac-48c7-b840-e4ed41972d38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:40.586875 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7395847c-f9ac-48c7-b840-e4ed41972d38 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:40.608344 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 181/342] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:28:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:40.609637 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c4c58-a6c3-4c05-b4df-e88a85c041c2 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:40.609928 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c4c58-a6c3-4c05-b4df-e88a85c041c2 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:40.610206 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c4c58-a6c3-4c05-b4df-e88a85c041c2 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:40.611491 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 182/343] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:40.621222 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c4c58-a6c3-4c05-b4df-e88a85c041c2 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:40.643284 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:40.688191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd5a3e-8c3a-4762-9c74-7e303bae1945 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:40.761023 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b694865-a27e-4634-af1d-33c3103c5dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:40.761633 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b694865-a27e-4634-af1d-33c3103c5dc0 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:40.761910 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b694865-a27e-4634-af1d-33c3103c5dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:40.762177 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b694865-a27e-4634-af1d-33c3103c5dc0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:40.818561 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c4c58-a6c3-4c05-b4df-e88a85c041c2 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:40.832567 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:40.886328 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2846edee-7bdb-479e-a220-3ff0fedecf92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:40.886942 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2846edee-7bdb-479e-a220-3ff0fedecf92 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:40.887221 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2846edee-7bdb-479e-a220-3ff0fedecf92 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:40.887490 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2846edee-7bdb-479e-a220-3ff0fedecf92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:40.894004 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b694865-a27e-4634-af1d-33c3103c5dc0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:40.895012 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b694865-a27e-4634-af1d-33c3103c5dc0 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:41.013208 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2846edee-7bdb-479e-a220-3ff0fedecf92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:41.015007 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2846edee-7bdb-479e-a220-3ff0fedecf92 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:41.040343 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:41.087391 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 162/344] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:28:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 634 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 22:28:41.088673 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00399229-1a6f-4b16-ae9f-abee0770fc6f None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:41.088984 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00399229-1a6f-4b16-ae9f-abee0770fc6f None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:41.089260 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00399229-1a6f-4b16-ae9f-abee0770fc6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:41.121664 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-00399229-1a6f-4b16-ae9f-abee0770fc6f None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:41.122854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-00399229-1a6f-4b16-ae9f-abee0770fc6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:41.190610 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 183/345] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:28:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:28:41.191968 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2846edee-7bdb-479e-a220-3ff0fedecf92 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:41.233093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:41.274285 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 163/346] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:41.275253 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00399229-1a6f-4b16-ae9f-abee0770fc6f None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:41.364079 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2846edee-7bdb-479e-a220-3ff0fedecf92 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:41.402923 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981bfef8-5cb9-4640-a991-f7f4e17794b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:41.403567 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981bfef8-5cb9-4640-a991-f7f4e17794b2 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:41.403846 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981bfef8-5cb9-4640-a991-f7f4e17794b2 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:41.404144 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981bfef8-5cb9-4640-a991-f7f4e17794b2 None admin] PATH_INFO: `/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:41.405318 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981bfef8-5cb9-4640-a991-f7f4e17794b2 None admin] RBAC: Authorizing `identity:get_project(project_id=bd224ec63c3748aa90cf3e984e32fa3f)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:41.406776 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981bfef8-5cb9-4640-a991-f7f4e17794b2 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:41.409334 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 184/347] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:28:41 2022] GET /identity/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:41.410577 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca25b9-57b5-4d62-9f4d-12ae755436df None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:41.410861 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca25b9-57b5-4d62-9f4d-12ae755436df None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:41.411132 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca25b9-57b5-4d62-9f4d-12ae755436df None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:41.412574 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 185/348] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:41.417572 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ca25b9-57b5-4d62-9f4d-12ae755436df None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:41.482532 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00399229-1a6f-4b16-ae9f-abee0770fc6f None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:41.497104 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:41.529113 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:41.557395 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ca25b9-57b5-4d62-9f4d-12ae755436df None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:41.560954 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddcafa2-4a1a-492b-adb4-9d9febd44511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:41.561586 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddcafa2-4a1a-492b-adb4-9d9febd44511 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:41.561864 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddcafa2-4a1a-492b-adb4-9d9febd44511 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:41.562137 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddcafa2-4a1a-492b-adb4-9d9febd44511 None admin] PATH_INFO: `/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:41.563256 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddcafa2-4a1a-492b-adb4-9d9febd44511 None admin] RBAC: Authorizing `identity:get_project(project_id=bd224ec63c3748aa90cf3e984e32fa3f)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:41.568934 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddcafa2-4a1a-492b-adb4-9d9febd44511 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:41.571001 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 164/349] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:28:41 2022] GET /identity/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f => generated 298 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:41.572238 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e89ec-bd32-4f0c-b4b7-47e70a26d809 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:41.572514 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e89ec-bd32-4f0c-b4b7-47e70a26d809 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:41.572804 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e89ec-bd32-4f0c-b4b7-47e70a26d809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:41.602701 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a2203-b03b-4d95-a18a-099d291f8e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:41.603353 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a2203-b03b-4d95-a18a-099d291f8e7d None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:41.604199 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-510e89ec-bd32-4f0c-b4b7-47e70a26d809 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:41.605044 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a2203-b03b-4d95-a18a-099d291f8e7d None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:41.605319 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a2203-b03b-4d95-a18a-099d291f8e7d None admin] PATH_INFO: `/v3/groups/8503ce49c0c04a70b1c75ce4471c5c97` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:41.606618 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-510e89ec-bd32-4f0c-b4b7-47e70a26d809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:41.619227 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622a2203-b03b-4d95-a18a-099d291f8e7d None admin] RBAC: Authorizing `identity:get_group(group_id=8503ce49c0c04a70b1c75ce4471c5c97)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:41.623003 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622a2203-b03b-4d95-a18a-099d291f8e7d None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:41.626193 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 186/350] 10.222.0.52 () {56 vars in 1180 bytes} [Mon Aug 29 22:28:41 2022] GET /identity/v3/groups/8503ce49c0c04a70b1c75ce4471c5c97 => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:41.626750 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a2203-b03b-4d95-a18a-099d291f8e7d None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:41.713638 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 165/351] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:41.714530 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e89ec-bd32-4f0c-b4b7-47e70a26d809 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:41.774283 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a2203-b03b-4d95-a18a-099d291f8e7d None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:41.823675 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a98905-c68a-4908-8f34-00f19f4bfa8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:41.824309 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a98905-c68a-4908-8f34-00f19f4bfa8f None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:41.824592 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a98905-c68a-4908-8f34-00f19f4bfa8f None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:41.824901 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a98905-c68a-4908-8f34-00f19f4bfa8f None admin] PATH_INFO: `/v3/groups/8503ce49c0c04a70b1c75ce4471c5c97` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:41.826029 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a98905-c68a-4908-8f34-00f19f4bfa8f None admin] RBAC: Authorizing `identity:get_group(group_id=8503ce49c0c04a70b1c75ce4471c5c97)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:41.827477 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a98905-c68a-4908-8f34-00f19f4bfa8f None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:41.829090 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 187/352] 10.222.0.52 () {56 vars in 1180 bytes} [Mon Aug 29 22:28:41 2022] GET /identity/v3/groups/8503ce49c0c04a70b1c75ce4471c5c97 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:41.829852 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a98905-c68a-4908-8f34-00f19f4bfa8f None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:41.842528 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:41.847489 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e89ec-bd32-4f0c-b4b7-47e70a26d809 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:41.856562 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:41.897732 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4afdf2-7917-45a8-bb64-1eaee1879c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:41.898794 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4afdf2-7917-45a8-bb64-1eaee1879c55 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:41.899143 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4afdf2-7917-45a8-bb64-1eaee1879c55 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:41.899514 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4afdf2-7917-45a8-bb64-1eaee1879c55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:41.900434 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4afdf2-7917-45a8-bb64-1eaee1879c55 None admin] RBAC: Adding query filter params (group.id=8503ce49c0c04a70b1c75ce4471c5c97, role.id=ae06bb653e3b4990a9ddaa1a649c7e06, scope.project.id=bd224ec63c3748aa90cf3e984e32fa3f) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:41.900774 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4afdf2-7917-45a8-bb64-1eaee1879c55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:41.903271 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4afdf2-7917-45a8-bb64-1eaee1879c55 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:41.912256 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 166/353] 10.222.0.52 () {56 vars in 1376 bytes} [Mon Aug 29 22:28:41 2022] GET /identity/v3/role_assignments?group.id=8503ce49c0c04a70b1c75ce4471c5c97&scope.project.id=bd224ec63c3748aa90cf3e984e32fa3f&role.id=ae06bb653e3b4990a9ddaa1a649c7e06 => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:41.914204 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86dcfa-ad89-435e-8db4-67f2aea24da6 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:41.914473 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86dcfa-ad89-435e-8db4-67f2aea24da6 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:41.914751 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86dcfa-ad89-435e-8db4-67f2aea24da6 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:41.916527 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 167/354] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:41.917812 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11aa76-8431-4c5f-84e8-c99d94ca3117 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:41.918473 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11aa76-8431-4c5f-84e8-c99d94ca3117 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:41.918746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11aa76-8431-4c5f-84e8-c99d94ca3117 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:41.921606 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 168/355] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:41.922698 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d11aa76-8431-4c5f-84e8-c99d94ca3117 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:41.971685 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a98905-c68a-4908-8f34-00f19f4bfa8f None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:42.032118 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868ec039-8e2e-4b9a-abd2-b21c0dee05a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:42.033454 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ec039-8e2e-4b9a-abd2-b21c0dee05a1 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:42.033706 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ec039-8e2e-4b9a-abd2-b21c0dee05a1 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:42.034583 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ec039-8e2e-4b9a-abd2-b21c0dee05a1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:42.050862 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868ec039-8e2e-4b9a-abd2-b21c0dee05a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:42.057913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-868ec039-8e2e-4b9a-abd2-b21c0dee05a1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:28:42.057913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:42.057913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:42.057913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:42.057913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:42.057913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:42.057913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:42.057913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:42.057913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:42.057913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:42.057913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:42.057913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:42.057913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:42.057913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:42.057913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:42.057913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:42.058880 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:42.058880 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:42.058880 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:42.058880 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:42.058880 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:42.058880 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:42.058880 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:42.058880 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:42.058880 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:42.058880 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:42.058880 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:42.058880 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:42.058880 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:42.058880 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:42.058880 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:42.058880 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:42.059689 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:42.059689 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:42.059689 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:42.059689 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:42.059689 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:42.059689 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:42.059689 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:42.059689 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:42.059689 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:42.059689 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:42.059689 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:28:42.059689 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:28:42.059689 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 188/356] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:28:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:42.060340 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e85635e-70f2-42c9-98c6-1032a4c97583 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:42.060340 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e85635e-70f2-42c9-98c6-1032a4c97583 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:42.061016 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e85635e-70f2-42c9-98c6-1032a4c97583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:42.071426 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d11aa76-8431-4c5f-84e8-c99d94ca3117 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:42.082621 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-5e85635e-70f2-42c9-98c6-1032a4c97583 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:42.083895 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-5e85635e-70f2-42c9-98c6-1032a4c97583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:42.110575 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff049114-a190-43f6-850f-bf7737b4463f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:42.112127 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff049114-a190-43f6-850f-bf7737b4463f None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:42.112589 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff049114-a190-43f6-850f-bf7737b4463f None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:42.113059 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff049114-a190-43f6-850f-bf7737b4463f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:42.113968 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff049114-a190-43f6-850f-bf7737b4463f None admin] RBAC: Adding query filter params (group.id=8503ce49c0c04a70b1c75ce4471c5c97, role.id=a6d1677ebe3a4355be5d67c5c78d83c7, scope.project.id=bd224ec63c3748aa90cf3e984e32fa3f) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:42.114520 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff049114-a190-43f6-850f-bf7737b4463f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:42.116298 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff049114-a190-43f6-850f-bf7737b4463f None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:42.125074 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 169/357] 10.222.0.52 () {56 vars in 1376 bytes} [Mon Aug 29 22:28:41 2022] GET /identity/v3/role_assignments?group.id=8503ce49c0c04a70b1c75ce4471c5c97&scope.project.id=bd224ec63c3748aa90cf3e984e32fa3f&role.id=a6d1677ebe3a4355be5d67c5c78d83c7 => generated 260 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:42.126559 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef639be-3708-47c3-a68a-e744adbd21b4 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:42.127781 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef639be-3708-47c3-a68a-e744adbd21b4 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:42.128781 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef639be-3708-47c3-a68a-e744adbd21b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:42.162875 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-aef639be-3708-47c3-a68a-e744adbd21b4 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:42.164772 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-aef639be-3708-47c3-a68a-e744adbd21b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:42.170848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:42.177800 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 189/358] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:42.178802 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e85635e-70f2-42c9-98c6-1032a4c97583 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:42.213162 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:42.265979 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 170/359] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:42.267298 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef639be-3708-47c3-a68a-e744adbd21b4 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:42.384936 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e85635e-70f2-42c9-98c6-1032a4c97583 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:42.386146 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef639be-3708-47c3-a68a-e744adbd21b4 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:42.426119 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a525625-dc5f-45d9-9aaf-b74a20485657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:42.427118 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a525625-dc5f-45d9-9aaf-b74a20485657 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:42.427597 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a525625-dc5f-45d9-9aaf-b74a20485657 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:42.428069 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a525625-dc5f-45d9-9aaf-b74a20485657 None admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:42.429063 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a525625-dc5f-45d9-9aaf-b74a20485657 None admin] RBAC: Adding query filter params (name=member) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:42.429567 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a525625-dc5f-45d9-9aaf-b74a20485657 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:42.431282 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a525625-dc5f-45d9-9aaf-b74a20485657 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:42.441814 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 171/360] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Aug 29 22:28:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:42.451151 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a525625-dc5f-45d9-9aaf-b74a20485657 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:42.453688 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf04b6a-4757-4932-b448-cb5006a1abc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:42.454387 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf04b6a-4757-4932-b448-cb5006a1abc8 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:42.454661 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf04b6a-4757-4932-b448-cb5006a1abc8 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:42.454932 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf04b6a-4757-4932-b448-cb5006a1abc8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:42.469385 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf04b6a-4757-4932-b448-cb5006a1abc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:42.475302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-bdf04b6a-4757-4932-b448-cb5006a1abc8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 22:28:42.475302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:42.475302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:42.475302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:42.475302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:42.475302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:42.475302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:42.475302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:42.475302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:42.475302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:42.475302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:42.475302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:42.475302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:42.475302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:42.475302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:42.475302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:42.476188 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:42.476188 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:42.476188 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:42.476188 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:42.476188 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:42.476188 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:42.476188 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:42.476188 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:42.476188 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:42.476188 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:42.476188 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:42.476188 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:42.476188 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:42.476188 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:42.476188 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:42.476188 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:42.481591 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:42.481591 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:42.481591 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:42.481591 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:42.481591 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:42.481591 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:42.481591 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:42.481591 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:42.481591 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:42.481591 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:42.481591 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 22:28:42.481591 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:28:42.481591 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 190/361] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:28:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:42.483244 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf04b6a-4757-4932-b448-cb5006a1abc8 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:42.495681 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:42.601893 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a525625-dc5f-45d9-9aaf-b74a20485657 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:42.632898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:42.649198 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d57c205-b72d-4ebe-9a66-f6802063cec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:42.650138 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57c205-b72d-4ebe-9a66-f6802063cec6 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:42.650618 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57c205-b72d-4ebe-9a66-f6802063cec6 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:42.651088 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57c205-b72d-4ebe-9a66-f6802063cec6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:42.740933 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d57c205-b72d-4ebe-9a66-f6802063cec6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:42.743091 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d57c205-b72d-4ebe-9a66-f6802063cec6 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:42.753174 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf04b6a-4757-4932-b448-cb5006a1abc8 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:42.795433 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9bcd9-0985-4182-9e87-51fdc434bfac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:42.796105 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9bcd9-0985-4182-9e87-51fdc434bfac None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:42.796387 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9bcd9-0985-4182-9e87-51fdc434bfac None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:42.796655 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9bcd9-0985-4182-9e87-51fdc434bfac None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:42.798194 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f9bcd9-0985-4182-9e87-51fdc434bfac None admin] RBAC: Adding query filter params (name=reader) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:42.798194 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f9bcd9-0985-4182-9e87-51fdc434bfac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:42.800121 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f9bcd9-0985-4182-9e87-51fdc434bfac None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:42.810701 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 191/362] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Aug 29 22:28:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:42.811527 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9bcd9-0985-4182-9e87-51fdc434bfac None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:42.833197 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:42.881688 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 172/363] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:28:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 22:28:42.882590 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d57c205-b72d-4ebe-9a66-f6802063cec6 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:42.939775 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9bcd9-0985-4182-9e87-51fdc434bfac None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:42.964009 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:42.986750 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e17e36-ff4a-4996-a413-cc80ce57b121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:42.987585 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e17e36-ff4a-4996-a413-cc80ce57b121 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:42.987868 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e17e36-ff4a-4996-a413-cc80ce57b121 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:42.988142 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e17e36-ff4a-4996-a413-cc80ce57b121 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:42.999710 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e17e36-ff4a-4996-a413-cc80ce57b121 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:43.003600 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d57c205-b72d-4ebe-9a66-f6802063cec6 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:43.005315 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-07e17e36-ff4a-4996-a413-cc80ce57b121 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:28:43.005315 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:43.005315 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:43.005315 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:43.005315 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:43.005315 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:43.005315 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:43.005315 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:43.005315 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:43.005315 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:43.005315 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:43.005315 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:43.005315 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:43.005315 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:43.005315 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:43.005315 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:43.006184 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:43.006184 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:43.006184 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:43.006184 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:43.006184 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:43.006184 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:43.006184 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:43.006184 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:43.006184 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:43.006184 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:43.006184 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:43.006184 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:43.006184 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:43.006184 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:43.006184 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:43.006184 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:43.006996 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:43.006996 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:43.006996 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:43.006996 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:43.006996 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:43.006996 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:43.006996 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:43.006996 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:43.006996 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:43.006996 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:43.006996 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:28:43.006996 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:28:43.006996 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 192/364] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Aug 29 22:28:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:43.007669 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e17e36-ff4a-4996-a413-cc80ce57b121 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:43.030997 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:43.044141 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cd9391-862d-4904-8a41-e15ced7a1e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:43.044800 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd9391-862d-4904-8a41-e15ced7a1e5e None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:43.045079 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd9391-862d-4904-8a41-e15ced7a1e5e None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:43.045343 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd9391-862d-4904-8a41-e15ced7a1e5e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:43.057746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cd9391-862d-4904-8a41-e15ced7a1e5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:43.063157 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-e9cd9391-862d-4904-8a41-e15ced7a1e5e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:28:43.063157 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:43.063157 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:43.063157 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:43.063157 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:43.063157 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:43.063157 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:43.063157 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:43.063157 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:43.063157 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:43.063157 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:43.063157 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:43.063157 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:43.063157 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:43.063157 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:43.063157 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:43.065854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:43.065854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:43.065854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:43.065854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:43.065854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:43.065854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:43.065854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:43.065854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:43.065854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:43.065854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:43.065854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:43.065854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:43.065854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:43.065854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:43.065854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:43.065854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:43.067787 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:43.067787 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:43.067787 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:43.067787 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:43.067787 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:43.067787 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:43.067787 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:43.067787 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:43.067787 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:43.067787 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:43.067787 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:28:43.067787 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:28:43.069278 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 173/365] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Aug 29 22:28:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:43.076831 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cd9391-862d-4904-8a41-e15ced7a1e5e None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:43.121028 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e17e36-ff4a-4996-a413-cc80ce57b121 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:43.158933 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7967594-cee3-4df5-b759-8e8c9b7fb466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:43.159584 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7967594-cee3-4df5-b759-8e8c9b7fb466 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:43.159866 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7967594-cee3-4df5-b759-8e8c9b7fb466 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:43.160139 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7967594-cee3-4df5-b759-8e8c9b7fb466 None admin] PATH_INFO: `/v3/users/2100ee3c17404419be6a50b9a6cef195` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:43.161369 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7967594-cee3-4df5-b759-8e8c9b7fb466 None admin] RBAC: Authorizing `identity:get_user(user_id=2100ee3c17404419be6a50b9a6cef195)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:43.163378 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7967594-cee3-4df5-b759-8e8c9b7fb466 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:43.165718 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 193/366] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:28:43 2022] GET /identity/v3/users/2100ee3c17404419be6a50b9a6cef195 => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:43.166893 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7967594-cee3-4df5-b759-8e8c9b7fb466 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:43.197373 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cd9391-862d-4904-8a41-e15ced7a1e5e None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:43.231263 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:43.244484 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23295b92-1940-4c65-8cf8-fbb0652dfead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:43.245180 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23295b92-1940-4c65-8cf8-fbb0652dfead None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:43.245456 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23295b92-1940-4c65-8cf8-fbb0652dfead None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:43.245723 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23295b92-1940-4c65-8cf8-fbb0652dfead None admin] PATH_INFO: `/v3/projects/baba6492015a48bfbca2121065b933ab` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:43.251245 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23295b92-1940-4c65-8cf8-fbb0652dfead None admin] RBAC: Authorizing `identity:get_project(project_id=baba6492015a48bfbca2121065b933ab)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:43.251245 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23295b92-1940-4c65-8cf8-fbb0652dfead None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:43.251245 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 174/367] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:28:43 2022] GET /identity/v3/projects/baba6492015a48bfbca2121065b933ab => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:43.256602 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23295b92-1940-4c65-8cf8-fbb0652dfead None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:43.296930 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7967594-cee3-4df5-b759-8e8c9b7fb466 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:43.320005 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:43.343822 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288337a0-8ca5-4e07-84af-132025d55701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:43.345243 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288337a0-8ca5-4e07-84af-132025d55701 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:43.345542 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288337a0-8ca5-4e07-84af-132025d55701 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:43.345871 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288337a0-8ca5-4e07-84af-132025d55701 None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:43.346544 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288337a0-8ca5-4e07-84af-132025d55701 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:43.346897 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288337a0-8ca5-4e07-84af-132025d55701 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:43.348342 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288337a0-8ca5-4e07-84af-132025d55701 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:43.359800 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 194/368] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:28:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:43.360890 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288337a0-8ca5-4e07-84af-132025d55701 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:43.379463 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23295b92-1940-4c65-8cf8-fbb0652dfead None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:43.426462 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc487fa0-c093-40eb-897e-309a8054bf11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:43.427099 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc487fa0-c093-40eb-897e-309a8054bf11 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:43.427539 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc487fa0-c093-40eb-897e-309a8054bf11 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:43.428012 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc487fa0-c093-40eb-897e-309a8054bf11 None admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:43.429342 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc487fa0-c093-40eb-897e-309a8054bf11 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:43.429789 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc487fa0-c093-40eb-897e-309a8054bf11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:43.431782 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc487fa0-c093-40eb-897e-309a8054bf11 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:43.444571 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 175/369] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:28:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:43.445764 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc487fa0-c093-40eb-897e-309a8054bf11 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:43.494903 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288337a0-8ca5-4e07-84af-132025d55701 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:43.509114 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:43.533525 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de895cd2-3eef-4ab8-ad93-fba7a69ff803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:43.534376 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de895cd2-3eef-4ab8-ad93-fba7a69ff803 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:43.534795 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de895cd2-3eef-4ab8-ad93-fba7a69ff803 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:43.535202 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de895cd2-3eef-4ab8-ad93-fba7a69ff803 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:43.536185 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de895cd2-3eef-4ab8-ad93-fba7a69ff803 None admin] RBAC: Adding query filter params (role.id=b9691ffb7caa429ab9a5876dd3dc2ba3, user.id=2100ee3c17404419be6a50b9a6cef195, scope.system=all) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:43.536630 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de895cd2-3eef-4ab8-ad93-fba7a69ff803 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:43.538426 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de895cd2-3eef-4ab8-ad93-fba7a69ff803 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:43.554248 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 195/370] 10.222.0.52 () {56 vars in 1308 bytes} [Mon Aug 29 22:28:43 2022] GET /identity/v3/role_assignments?user.id=2100ee3c17404419be6a50b9a6cef195&scope.system=all&role.id=b9691ffb7caa429ab9a5876dd3dc2ba3 => generated 510 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:43.561539 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc487fa0-c093-40eb-897e-309a8054bf11 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:43.565801 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de895cd2-3eef-4ab8-ad93-fba7a69ff803 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:43.578193 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:43.603923 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b331b8-df47-4b40-81ac-b874c9db664f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:43.605849 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b331b8-df47-4b40-81ac-b874c9db664f None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:43.606465 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b331b8-df47-4b40-81ac-b874c9db664f None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:43.606940 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b331b8-df47-4b40-81ac-b874c9db664f None admin] PATH_INFO: `/v3/groups/8503ce49c0c04a70b1c75ce4471c5c97` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:43.608459 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b331b8-df47-4b40-81ac-b874c9db664f None admin] RBAC: Authorizing `identity:get_group(group_id=8503ce49c0c04a70b1c75ce4471c5c97)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:43.610360 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b331b8-df47-4b40-81ac-b874c9db664f None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:43.612859 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 176/371] 10.222.0.52 () {56 vars in 1180 bytes} [Mon Aug 29 22:28:43 2022] GET /identity/v3/groups/8503ce49c0c04a70b1c75ce4471c5c97 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:43.613901 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b331b8-df47-4b40-81ac-b874c9db664f None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:43.731544 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de895cd2-3eef-4ab8-ad93-fba7a69ff803 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:43.733862 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b331b8-df47-4b40-81ac-b874c9db664f None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:43.775550 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd86f7-4df1-4f78-9e1a-1040a4f13246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:43.775890 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36539406-fa8e-4e0b-bf69-f1c8c1a84fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:43.776413 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd86f7-4df1-4f78-9e1a-1040a4f13246 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:43.776884 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd86f7-4df1-4f78-9e1a-1040a4f13246 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:43.777008 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36539406-fa8e-4e0b-bf69-f1c8c1a84fdd None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:43.777341 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36539406-fa8e-4e0b-bf69-f1c8c1a84fdd None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:43.777578 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd86f7-4df1-4f78-9e1a-1040a4f13246 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:43.778740 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36539406-fa8e-4e0b-bf69-f1c8c1a84fdd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:43.828446 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:43.838301 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:43.858634 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36539406-fa8e-4e0b-bf69-f1c8c1a84fdd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:43.859042 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd86f7-4df1-4f78-9e1a-1040a4f13246 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:43.860103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36539406-fa8e-4e0b-bf69-f1c8c1a84fdd None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:43.860477 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd86f7-4df1-4f78-9e1a-1040a4f13246 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:43.980458 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 177/372] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:28:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:28:43.981596 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36539406-fa8e-4e0b-bf69-f1c8c1a84fdd None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:43.981728 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 196/373] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:28:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:28:43.988267 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd86f7-4df1-4f78-9e1a-1040a4f13246 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:44.071386 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:44.087524 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:44.111062 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36539406-fa8e-4e0b-bf69-f1c8c1a84fdd None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:44.112856 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd86f7-4df1-4f78-9e1a-1040a4f13246 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:44.154794 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2d593b-77f2-4baf-aa16-41c4b4bdf9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:44.155138 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a20453a-4c97-4914-acb8-76a59acd3eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:44.155932 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a20453a-4c97-4914-acb8-76a59acd3eb9 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:44.156363 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a20453a-4c97-4914-acb8-76a59acd3eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:44.156481 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d593b-77f2-4baf-aa16-41c4b4bdf9ef None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:44.156748 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d593b-77f2-4baf-aa16-41c4b4bdf9ef None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:44.156918 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a20453a-4c97-4914-acb8-76a59acd3eb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:44.157320 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d593b-77f2-4baf-aa16-41c4b4bdf9ef None admin] PATH_INFO: `/v3/projects/baba6492015a48bfbca2121065b933ab` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:44.157655 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a20453a-4c97-4914-acb8-76a59acd3eb9 None admin] RBAC: Adding query filter params (group.id=8503ce49c0c04a70b1c75ce4471c5c97, role.id=ae06bb653e3b4990a9ddaa1a649c7e06, scope.project.id=baba6492015a48bfbca2121065b933ab) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:44.157974 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a20453a-4c97-4914-acb8-76a59acd3eb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:44.159608 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2d593b-77f2-4baf-aa16-41c4b4bdf9ef None admin] RBAC: Authorizing `identity:get_project(project_id=baba6492015a48bfbca2121065b933ab)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:44.160159 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a20453a-4c97-4914-acb8-76a59acd3eb9 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:44.161389 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2d593b-77f2-4baf-aa16-41c4b4bdf9ef None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:44.163636 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 197/374] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:28:43 2022] GET /identity/v3/projects/baba6492015a48bfbca2121065b933ab => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:44.164557 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2d593b-77f2-4baf-aa16-41c4b4bdf9ef None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:44.168740 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 178/375] 10.222.0.52 () {56 vars in 1376 bytes} [Mon Aug 29 22:28:43 2022] GET /identity/v3/role_assignments?group.id=8503ce49c0c04a70b1c75ce4471c5c97&scope.project.id=baba6492015a48bfbca2121065b933ab&role.id=ae06bb653e3b4990a9ddaa1a649c7e06 => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:44.172284 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a20453a-4c97-4914-acb8-76a59acd3eb9 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:44.301917 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2d593b-77f2-4baf-aa16-41c4b4bdf9ef None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:44.324277 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a20453a-4c97-4914-acb8-76a59acd3eb9 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:44.350324 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac00024-05ad-4509-b2d0-a486c29026ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:44.351102 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac00024-05ad-4509-b2d0-a486c29026ef None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:44.351385 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac00024-05ad-4509-b2d0-a486c29026ef None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:44.351924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac00024-05ad-4509-b2d0-a486c29026ef None admin] PATH_INFO: `/v3/projects/eb4839200e8e45dfa0acd9e9d8f433bc` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:44.352923 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac00024-05ad-4509-b2d0-a486c29026ef None admin] RBAC: Authorizing `identity:get_project(project_id=eb4839200e8e45dfa0acd9e9d8f433bc)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:44.354382 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac00024-05ad-4509-b2d0-a486c29026ef None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:44.356845 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 198/376] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:28:44 2022] GET /identity/v3/projects/eb4839200e8e45dfa0acd9e9d8f433bc => generated 344 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:44.361929 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac00024-05ad-4509-b2d0-a486c29026ef None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:44.369016 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:44.381694 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151cd378-b679-4ee1-bd91-de1715d2ed21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:44.382672 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151cd378-b679-4ee1-bd91-de1715d2ed21 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:44.383139 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151cd378-b679-4ee1-bd91-de1715d2ed21 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:44.383609 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151cd378-b679-4ee1-bd91-de1715d2ed21 None admin] PATH_INFO: `/v3/groups/8503ce49c0c04a70b1c75ce4471c5c97` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:44.385217 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151cd378-b679-4ee1-bd91-de1715d2ed21 None admin] RBAC: Authorizing `identity:get_group(group_id=8503ce49c0c04a70b1c75ce4471c5c97)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:44.393501 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151cd378-b679-4ee1-bd91-de1715d2ed21 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:44.393501 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 179/377] 10.222.0.52 () {56 vars in 1180 bytes} [Mon Aug 29 22:28:44 2022] GET /identity/v3/groups/8503ce49c0c04a70b1c75ce4471c5c97 => generated 226 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:44.398670 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151cd378-b679-4ee1-bd91-de1715d2ed21 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:44.440337 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:44.502785 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac00024-05ad-4509-b2d0-a486c29026ef None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:44.544472 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4e2afa-9741-4d52-98e0-6e678dff11ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:44.545502 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e2afa-9741-4d52-98e0-6e678dff11ca None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:44.545988 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e2afa-9741-4d52-98e0-6e678dff11ca None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:44.546462 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e2afa-9741-4d52-98e0-6e678dff11ca None admin] PATH_INFO: `/v3/groups/0d92f1ba2d474a22a92ef4a2a01f2b9d` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:44.553162 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4e2afa-9741-4d52-98e0-6e678dff11ca None admin] RBAC: Authorizing `identity:get_group(group_id=0d92f1ba2d474a22a92ef4a2a01f2b9d)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:44.554876 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4e2afa-9741-4d52-98e0-6e678dff11ca None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:44.555907 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151cd378-b679-4ee1-bd91-de1715d2ed21 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:44.557125 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 199/378] 10.222.0.52 () {56 vars in 1180 bytes} [Mon Aug 29 22:28:44 2022] GET /identity/v3/groups/0d92f1ba2d474a22a92ef4a2a01f2b9d => generated 229 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:28:44.558510 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc00c5-4a8a-42fa-817a-4f96ecbd0d0c None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:44.559006 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc00c5-4a8a-42fa-817a-4f96ecbd0d0c None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:44.559485 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc00c5-4a8a-42fa-817a-4f96ecbd0d0c None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:44.561287 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 200/379] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:44.563795 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc00c5-4a8a-42fa-817a-4f96ecbd0d0c None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:44.614968 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b9ebe8-2e1a-4cc2-a0a0-24fb12e81090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:44.616763 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9ebe8-2e1a-4cc2-a0a0-24fb12e81090 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:44.617192 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9ebe8-2e1a-4cc2-a0a0-24fb12e81090 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:44.617624 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9ebe8-2e1a-4cc2-a0a0-24fb12e81090 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:44.619277 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b9ebe8-2e1a-4cc2-a0a0-24fb12e81090 None admin] RBAC: Adding query filter params (group.id=8503ce49c0c04a70b1c75ce4471c5c97, role.id=a6d1677ebe3a4355be5d67c5c78d83c7, scope.project.id=baba6492015a48bfbca2121065b933ab) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:44.619755 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b9ebe8-2e1a-4cc2-a0a0-24fb12e81090 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:44.622290 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b9ebe8-2e1a-4cc2-a0a0-24fb12e81090 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:44.633477 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 180/380] 10.222.0.52 () {56 vars in 1376 bytes} [Mon Aug 29 22:28:44 2022] GET /identity/v3/role_assignments?group.id=8503ce49c0c04a70b1c75ce4471c5c97&scope.project.id=baba6492015a48bfbca2121065b933ab&role.id=a6d1677ebe3a4355be5d67c5c78d83c7 => generated 260 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:44.635851 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb63d7e-e540-407f-9408-de72b282f107 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:44.636292 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb63d7e-e540-407f-9408-de72b282f107 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:44.636775 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb63d7e-e540-407f-9408-de72b282f107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:44.660383 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-6eb63d7e-e540-407f-9408-de72b282f107 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:44.662497 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-6eb63d7e-e540-407f-9408-de72b282f107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:44.674080 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:44.749598 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc00c5-4a8a-42fa-817a-4f96ecbd0d0c None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:44.756085 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 181/381] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:44.811834 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3957c0b-12bc-4593-a818-952066e8e1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:44.812443 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3957c0b-12bc-4593-a818-952066e8e1fa None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:44.812742 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3957c0b-12bc-4593-a818-952066e8e1fa None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:44.813019 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3957c0b-12bc-4593-a818-952066e8e1fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:44.813754 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3957c0b-12bc-4593-a818-952066e8e1fa None admin] RBAC: Adding query filter params (group.id=0d92f1ba2d474a22a92ef4a2a01f2b9d, role.id=882f35bd847a406eb2222aeafc517a46, scope.project.id=eb4839200e8e45dfa0acd9e9d8f433bc) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:44.814065 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3957c0b-12bc-4593-a818-952066e8e1fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:44.816159 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3957c0b-12bc-4593-a818-952066e8e1fa None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:44.825299 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 201/382] 10.222.0.52 () {56 vars in 1376 bytes} [Mon Aug 29 22:28:44 2022] GET /identity/v3/role_assignments?group.id=0d92f1ba2d474a22a92ef4a2a01f2b9d&scope.project.id=eb4839200e8e45dfa0acd9e9d8f433bc&role.id=882f35bd847a406eb2222aeafc517a46 => generated 260 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:28:44.997849 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4942d668-6162-4ced-aac6-dc45ea46df85 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:44.998236 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4942d668-6162-4ced-aac6-dc45ea46df85 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:44.998410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4942d668-6162-4ced-aac6-dc45ea46df85 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:45.000009 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 182/383] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:45.006630 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902672e-7746-4abb-a2bb-9e0c44c3958e None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:45.006932 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902672e-7746-4abb-a2bb-9e0c44c3958e None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:45.007258 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902672e-7746-4abb-a2bb-9e0c44c3958e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:45.034068 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-3902672e-7746-4abb-a2bb-9e0c44c3958e None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:45.035286 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-3902672e-7746-4abb-a2bb-9e0c44c3958e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:45.071681 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4942d668-6162-4ced-aac6-dc45ea46df85 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:45.089101 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:45.129646 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 202/384] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:45.201513 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4942d668-6162-4ced-aac6-dc45ea46df85 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:45.237866 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f7741-721b-4753-82d2-e8c6f2910845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:45.238508 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f7741-721b-4753-82d2-e8c6f2910845 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:45.238791 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f7741-721b-4753-82d2-e8c6f2910845 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:45.239065 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f7741-721b-4753-82d2-e8c6f2910845 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:45.250064 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4f7741-721b-4753-82d2-e8c6f2910845 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:45.255533 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-ef4f7741-721b-4753-82d2-e8c6f2910845 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:28:45.255533 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:45.255533 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:45.255533 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:45.255533 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:45.255533 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:45.255533 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:45.255533 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:45.255533 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:45.255533 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:45.255533 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:45.255533 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:45.255533 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:45.255533 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:45.255533 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:45.255533 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:45.256485 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:45.256485 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:45.256485 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:45.256485 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:45.256485 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:45.256485 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:45.256485 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:45.256485 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:45.256485 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:45.256485 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:45.256485 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:45.256485 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:45.256485 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:45.256485 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:45.256485 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:45.256485 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:45.257361 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:45.257361 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:45.257361 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:45.257361 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:45.257361 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:45.257361 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:45.257361 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:45.257361 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:45.257361 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:45.257361 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:45.257361 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:28:45.257361 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:28:45.257361 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 183/385] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:28:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:45.266942 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3902672e-7746-4abb-a2bb-9e0c44c3958e None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:45.337109 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:45.377491 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3902672e-7746-4abb-a2bb-9e0c44c3958e None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:45.415177 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a901b-c346-4f03-9750-74a6e15c7b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:45.415799 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a901b-c346-4f03-9750-74a6e15c7b68 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:45.416080 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a901b-c346-4f03-9750-74a6e15c7b68 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:45.416359 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a901b-c346-4f03-9750-74a6e15c7b68 None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:45.417062 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49a901b-c346-4f03-9750-74a6e15c7b68 None admin] RBAC: Adding query filter params (name=member) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:45.417369 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49a901b-c346-4f03-9750-74a6e15c7b68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:45.418767 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49a901b-c346-4f03-9750-74a6e15c7b68 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:45.428851 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 203/386] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Aug 29 22:28:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:45.438112 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f7741-721b-4753-82d2-e8c6f2910845 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:45.513421 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:45.560308 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f7741-721b-4753-82d2-e8c6f2910845 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:45.605293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8e4a6e-84c3-4a27-99cd-1e8fe1d6eb7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:45.606273 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e4a6e-84c3-4a27-99cd-1e8fe1d6eb7e None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:45.606769 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e4a6e-84c3-4a27-99cd-1e8fe1d6eb7e None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:45.607264 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e4a6e-84c3-4a27-99cd-1e8fe1d6eb7e None admin] PATH_INFO: `/v3/groups/8503ce49c0c04a70b1c75ce4471c5c97` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:45.608686 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8e4a6e-84c3-4a27-99cd-1e8fe1d6eb7e None admin] RBAC: Authorizing `identity:get_group(group_id=8503ce49c0c04a70b1c75ce4471c5c97)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:45.610352 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8e4a6e-84c3-4a27-99cd-1e8fe1d6eb7e None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:45.612768 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 184/387] 10.222.0.52 () {56 vars in 1180 bytes} [Mon Aug 29 22:28:45 2022] GET /identity/v3/groups/8503ce49c0c04a70b1c75ce4471c5c97 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:45.618714 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a901b-c346-4f03-9750-74a6e15c7b68 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:45.686478 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8e4a6e-84c3-4a27-99cd-1e8fe1d6eb7e None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:45.756921 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a901b-c346-4f03-9750-74a6e15c7b68 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:45.796064 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:45.811944 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab24eb-66dc-4ffd-9bfc-a84836ef9377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:45.812862 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab24eb-66dc-4ffd-9bfc-a84836ef9377 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:45.813390 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab24eb-66dc-4ffd-9bfc-a84836ef9377 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:45.813822 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab24eb-66dc-4ffd-9bfc-a84836ef9377 None admin] PATH_INFO: `/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:45.819064 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8e4a6e-84c3-4a27-99cd-1e8fe1d6eb7e None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:45.821417 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ab24eb-66dc-4ffd-9bfc-a84836ef9377 None admin] RBAC: Authorizing `identity:get_project(project_id=bd224ec63c3748aa90cf3e984e32fa3f)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:45.822902 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ab24eb-66dc-4ffd-9bfc-a84836ef9377 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:45.825069 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 204/388] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:28:45 2022] GET /identity/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:45.831437 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab24eb-66dc-4ffd-9bfc-a84836ef9377 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:45.867205 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17684e57-3513-4ae2-9463-ae180d6de6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:45.867851 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17684e57-3513-4ae2-9463-ae180d6de6f5 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:45.868131 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17684e57-3513-4ae2-9463-ae180d6de6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:45.868401 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17684e57-3513-4ae2-9463-ae180d6de6f5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:45.874280 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:45.889462 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17684e57-3513-4ae2-9463-ae180d6de6f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:45.895093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-17684e57-3513-4ae2-9463-ae180d6de6f5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:28:45.895093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:45.895093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:45.895093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:45.895093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:45.895093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:45.895093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:45.895093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:45.895093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:45.895093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:45.895093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:45.895093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:45.895093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:45.895093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:45.895093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:45.895093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:45.895983 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:45.895983 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:45.895983 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:45.895983 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:45.895983 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:45.895983 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:45.895983 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:45.895983 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:45.895983 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:45.895983 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:45.895983 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:45.895983 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:45.895983 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:45.895983 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:45.895983 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:45.895983 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:45.896840 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:45.896840 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:45.896840 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:45.896840 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:45.896840 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:45.896840 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:45.896840 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:45.896840 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:45.896840 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:45.896840 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:45.896840 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:28:45.896840 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:28:45.897606 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 185/389] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Aug 29 22:28:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:45.908877 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17684e57-3513-4ae2-9463-ae180d6de6f5 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:46.035562 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab24eb-66dc-4ffd-9bfc-a84836ef9377 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:46.045127 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17684e57-3513-4ae2-9463-ae180d6de6f5 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:46.087696 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bee504-969f-4385-8583-9dd0b02aca50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:46.088560 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bee504-969f-4385-8583-9dd0b02aca50 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:46.089055 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bee504-969f-4385-8583-9dd0b02aca50 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:46.089575 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bee504-969f-4385-8583-9dd0b02aca50 None admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:46.090654 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bee504-969f-4385-8583-9dd0b02aca50 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:46.091460 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bee504-969f-4385-8583-9dd0b02aca50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:46.093407 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bee504-969f-4385-8583-9dd0b02aca50 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:46.100449 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e90cc55-0e32-4150-bf2f-4c8713aaf064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:46.101146 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e90cc55-0e32-4150-bf2f-4c8713aaf064 None admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:46.101426 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e90cc55-0e32-4150-bf2f-4c8713aaf064 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:46.101698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e90cc55-0e32-4150-bf2f-4c8713aaf064 None admin] PATH_INFO: `/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f/groups/8503ce49c0c04a70b1c75ce4471c5c97/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:46.105997 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e90cc55-0e32-4150-bf2f-4c8713aaf064 None admin] RBAC: Authorizing `identity:create_grant(project_id=bd224ec63c3748aa90cf3e984e32fa3f, group_id=8503ce49c0c04a70b1c75ce4471c5c97, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:46.106815 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 186/390] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:28:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:46.107506 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e90cc55-0e32-4150-bf2f-4c8713aaf064 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:46.115876 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bee504-969f-4385-8583-9dd0b02aca50 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:46.185456 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:46.202887 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 205/391] 10.222.0.52 () {58 vars in 1442 bytes} [Mon Aug 29 22:28:45 2022] PUT /identity/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f/groups/8503ce49c0c04a70b1c75ce4471c5c97/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:28:46.247716 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bee504-969f-4385-8583-9dd0b02aca50 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:46.301845 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f58406-805b-4663-aee2-e4cc92c3f4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:46.302508 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f58406-805b-4663-aee2-e4cc92c3f4dd None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:46.302786 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f58406-805b-4663-aee2-e4cc92c3f4dd None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:46.303062 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f58406-805b-4663-aee2-e4cc92c3f4dd None admin] PATH_INFO: `/v3/groups/8503ce49c0c04a70b1c75ce4471c5c97` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:46.304230 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f58406-805b-4663-aee2-e4cc92c3f4dd None admin] RBAC: Authorizing `identity:get_group(group_id=8503ce49c0c04a70b1c75ce4471c5c97)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:46.305713 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f58406-805b-4663-aee2-e4cc92c3f4dd None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:46.307285 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 187/392] 10.222.0.52 () {56 vars in 1180 bytes} [Mon Aug 29 22:28:46 2022] GET /identity/v3/groups/8503ce49c0c04a70b1c75ce4471c5c97 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:46.313120 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e90cc55-0e32-4150-bf2f-4c8713aaf064 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:46.397703 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:46.461957 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e90cc55-0e32-4150-bf2f-4c8713aaf064 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:46.507335 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753f0463-0f13-4de0-988e-bbcf89742ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:46.508384 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f0463-0f13-4de0-988e-bbcf89742ef5 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:46.508934 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f0463-0f13-4de0-988e-bbcf89742ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:46.509420 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f0463-0f13-4de0-988e-bbcf89742ef5 None admin] PATH_INFO: `/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:46.510781 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753f0463-0f13-4de0-988e-bbcf89742ef5 None admin] RBAC: Authorizing `identity:get_project(project_id=bd224ec63c3748aa90cf3e984e32fa3f)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:46.512410 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753f0463-0f13-4de0-988e-bbcf89742ef5 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:46.514632 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 206/393] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:28:46 2022] GET /identity/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:46.520663 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f58406-805b-4663-aee2-e4cc92c3f4dd None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:46.677944 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f58406-805b-4663-aee2-e4cc92c3f4dd None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:46.714240 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:46.724867 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e586bb2-f8f5-493f-b06f-ed7aa1c61987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:46.725843 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e586bb2-f8f5-493f-b06f-ed7aa1c61987 None admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:46.726331 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e586bb2-f8f5-493f-b06f-ed7aa1c61987 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:46.728552 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e586bb2-f8f5-493f-b06f-ed7aa1c61987 None admin] PATH_INFO: `/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f/groups/8503ce49c0c04a70b1c75ce4471c5c97/roles/a6d1677ebe3a4355be5d67c5c78d83c7` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:46.729286 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e586bb2-f8f5-493f-b06f-ed7aa1c61987 None admin] RBAC: Authorizing `identity:create_grant(project_id=bd224ec63c3748aa90cf3e984e32fa3f, group_id=8503ce49c0c04a70b1c75ce4471c5c97, role_id=a6d1677ebe3a4355be5d67c5c78d83c7)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:46.731173 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e586bb2-f8f5-493f-b06f-ed7aa1c61987 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:46.813999 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 188/394] 10.222.0.52 () {58 vars in 1442 bytes} [Mon Aug 29 22:28:46 2022] PUT /identity/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f/groups/8503ce49c0c04a70b1c75ce4471c5c97/roles/a6d1677ebe3a4355be5d67c5c78d83c7 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:28:47.067695 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d22fdf-9a14-4858-b35d-a2a04e1c7f47 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:47.068248 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d22fdf-9a14-4858-b35d-a2a04e1c7f47 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:47.068248 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d22fdf-9a14-4858-b35d-a2a04e1c7f47 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:47.069914 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 207/395] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:47.076350 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de741ebe-55b2-4f55-8e8c-ddbd74f4b3a4 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:47.076704 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de741ebe-55b2-4f55-8e8c-ddbd74f4b3a4 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:47.076989 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de741ebe-55b2-4f55-8e8c-ddbd74f4b3a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:47.105090 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-de741ebe-55b2-4f55-8e8c-ddbd74f4b3a4 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:47.106302 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-de741ebe-55b2-4f55-8e8c-ddbd74f4b3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:47.194209 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 189/396] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:47.216698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd76488f-0f9b-4a50-a10f-c3b79e2dd239 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:47.217014 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd76488f-0f9b-4a50-a10f-c3b79e2dd239 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:47.217293 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd76488f-0f9b-4a50-a10f-c3b79e2dd239 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:47.218800 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 208/397] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:47.224628 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f5728-d717-42da-8017-acac0ba7c9dd None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:47.224946 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f5728-d717-42da-8017-acac0ba7c9dd None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:47.225221 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f5728-d717-42da-8017-acac0ba7c9dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:47.255300 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-969f5728-d717-42da-8017-acac0ba7c9dd None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:47.256493 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-969f5728-d717-42da-8017-acac0ba7c9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:47.373935 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 190/398] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:47.399712 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e806ca75-c412-4c74-a3ae-fd6b28cab93c None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:47.400050 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e806ca75-c412-4c74-a3ae-fd6b28cab93c None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:47.400302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e806ca75-c412-4c74-a3ae-fd6b28cab93c None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:47.403717 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 209/399] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:47.410003 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79360f-0cec-48f5-a3de-8bdc9b4d2c9f None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:47.410298 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79360f-0cec-48f5-a3de-8bdc9b4d2c9f None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:47.410580 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79360f-0cec-48f5-a3de-8bdc9b4d2c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:47.437050 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-5f79360f-0cec-48f5-a3de-8bdc9b4d2c9f None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:47.438258 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-5f79360f-0cec-48f5-a3de-8bdc9b4d2c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:47.503960 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:47.521613 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e806ca75-c412-4c74-a3ae-fd6b28cab93c None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:47.557526 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 191/400] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:47.677089 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e806ca75-c412-4c74-a3ae-fd6b28cab93c None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:47.698753 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:47.714127 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423266a4-0132-4999-978a-c0ef0e665721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:47.714760 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423266a4-0132-4999-978a-c0ef0e665721 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:47.715032 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423266a4-0132-4999-978a-c0ef0e665721 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:47.715299 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423266a4-0132-4999-978a-c0ef0e665721 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:47.727502 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423266a4-0132-4999-978a-c0ef0e665721 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:47.732829 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-423266a4-0132-4999-978a-c0ef0e665721 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:28:47.732829 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:47.732829 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:47.732829 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:47.732829 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:47.732829 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:47.732829 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:47.732829 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:47.732829 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:47.732829 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:47.732829 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:47.732829 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:47.732829 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:47.732829 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:47.732829 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:47.732829 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:47.733906 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:47.733906 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:47.733906 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:47.733906 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:47.733906 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:47.733906 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:47.733906 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:47.733906 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:47.733906 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:47.733906 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:47.733906 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:47.733906 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:47.733906 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:47.733906 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:47.733906 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:47.733906 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:47.734885 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:47.734885 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:47.734885 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:47.734885 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:47.734885 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:47.734885 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:47.734885 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:47.734885 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:47.734885 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:47.734885 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:47.734885 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:28:47.734885 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:28:47.737832 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 210/401] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:28:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:28:47.744025 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f79360f-0cec-48f5-a3de-8bdc9b4d2c9f None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:47.854023 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423266a4-0132-4999-978a-c0ef0e665721 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:47.885321 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f79360f-0cec-48f5-a3de-8bdc9b4d2c9f None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:47.925345 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9a8bb-b98e-4328-9004-6eceb3e6f3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:47.926044 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9a8bb-b98e-4328-9004-6eceb3e6f3ee None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:47.926397 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9a8bb-b98e-4328-9004-6eceb3e6f3ee None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:47.926748 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9a8bb-b98e-4328-9004-6eceb3e6f3ee None admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:47.927647 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b9a8bb-b98e-4328-9004-6eceb3e6f3ee None admin] RBAC: Adding query filter params (name=member) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:47.927965 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b9a8bb-b98e-4328-9004-6eceb3e6f3ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:47.929523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b9a8bb-b98e-4328-9004-6eceb3e6f3ee None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:47.938585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 192/402] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Aug 29 22:28:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:28:47.939790 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9a8bb-b98e-4328-9004-6eceb3e6f3ee None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:47.965344 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423266a4-0132-4999-978a-c0ef0e665721 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:47.984677 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:48.002840 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51774ddb-ab2b-443b-b361-933d5c63dfa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:48.003754 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51774ddb-ab2b-443b-b361-933d5c63dfa5 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:48.004234 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51774ddb-ab2b-443b-b361-933d5c63dfa5 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:48.004674 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51774ddb-ab2b-443b-b361-933d5c63dfa5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:48.015904 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51774ddb-ab2b-443b-b361-933d5c63dfa5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:48.022465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-51774ddb-ab2b-443b-b361-933d5c63dfa5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:28:48.022465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:48.022465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:48.022465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:48.022465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:48.022465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:48.022465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:48.022465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:48.022465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:48.022465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:48.022465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:48.022465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:48.022465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:48.022465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:48.022465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:48.022465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:48.023833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:48.023833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:48.023833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:48.023833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:48.023833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:48.023833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:48.023833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:48.023833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:48.023833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:48.023833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:48.023833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:48.023833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:48.023833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:48.023833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:48.023833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:48.023833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:48.024604 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:48.024604 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:48.024604 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:48.024604 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:48.024604 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:48.024604 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:48.024604 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:48.024604 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:48.024604 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:48.024604 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:48.024604 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:28:48.024604 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:28:48.031700 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 211/403] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Aug 29 22:28:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:28:48.032900 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51774ddb-ab2b-443b-b361-933d5c63dfa5 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:48.071546 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9a8bb-b98e-4328-9004-6eceb3e6f3ee None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:48.082477 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:48.117555 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db61d385-244c-41ad-bf3c-f3d658adcdf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:48.118474 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61d385-244c-41ad-bf3c-f3d658adcdf7 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:48.118759 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61d385-244c-41ad-bf3c-f3d658adcdf7 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:48.119073 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61d385-244c-41ad-bf3c-f3d658adcdf7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:48.136214 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db61d385-244c-41ad-bf3c-f3d658adcdf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:48.143816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-db61d385-244c-41ad-bf3c-f3d658adcdf7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:28:48.143816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:48.143816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:48.143816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:48.143816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:48.143816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:48.143816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:48.143816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:48.143816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:48.143816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:48.143816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:48.143816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:48.143816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:48.143816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:48.143816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:48.143816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:48.144749 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:48.144749 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:48.144749 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:48.144749 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:48.144749 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:48.144749 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:48.144749 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:48.144749 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:48.144749 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:48.144749 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:48.144749 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:48.144749 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:48.144749 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:48.144749 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:48.144749 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:48.144749 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:48.146137 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:48.146137 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:48.146137 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:48.146137 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:48.146137 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:48.146137 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:48.146137 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:48.146137 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:48.146137 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:48.146137 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:48.146137 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:28:48.146137 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:28:48.146137 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 193/404] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Aug 29 22:28:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:48.146137 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db61d385-244c-41ad-bf3c-f3d658adcdf7 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:48.165303 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51774ddb-ab2b-443b-b361-933d5c63dfa5 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:48.209369 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da369097-3ac8-4393-af80-62b45de86bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:48.210492 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da369097-3ac8-4393-af80-62b45de86bc1 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:48.210951 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da369097-3ac8-4393-af80-62b45de86bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:48.211538 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da369097-3ac8-4393-af80-62b45de86bc1 None admin] PATH_INFO: `/v3/groups/8503ce49c0c04a70b1c75ce4471c5c97` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:48.213146 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da369097-3ac8-4393-af80-62b45de86bc1 None admin] RBAC: Authorizing `identity:get_group(group_id=8503ce49c0c04a70b1c75ce4471c5c97)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:48.214731 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da369097-3ac8-4393-af80-62b45de86bc1 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:48.217239 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 212/405] 10.222.0.52 () {56 vars in 1180 bytes} [Mon Aug 29 22:28:48 2022] GET /identity/v3/groups/8503ce49c0c04a70b1c75ce4471c5c97 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:48.218490 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da369097-3ac8-4393-af80-62b45de86bc1 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:48.276901 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:48.282677 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db61d385-244c-41ad-bf3c-f3d658adcdf7 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:48.335383 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739d7c1a-fd8f-4424-a2a1-41b8750445fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:48.336178 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739d7c1a-fd8f-4424-a2a1-41b8750445fb None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:48.336533 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739d7c1a-fd8f-4424-a2a1-41b8750445fb None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:48.336931 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739d7c1a-fd8f-4424-a2a1-41b8750445fb None admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:48.337700 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739d7c1a-fd8f-4424-a2a1-41b8750445fb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:48.338214 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739d7c1a-fd8f-4424-a2a1-41b8750445fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:48.339798 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739d7c1a-fd8f-4424-a2a1-41b8750445fb None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:48.349940 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 194/406] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:28:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:48.350902 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da369097-3ac8-4393-af80-62b45de86bc1 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:48.351021 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739d7c1a-fd8f-4424-a2a1-41b8750445fb None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:48.362589 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:48.397076 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb4c68-b95f-4f32-8adc-4c5f795ab323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:48.397968 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb4c68-b95f-4f32-8adc-4c5f795ab323 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:48.398400 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb4c68-b95f-4f32-8adc-4c5f795ab323 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:48.398810 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb4c68-b95f-4f32-8adc-4c5f795ab323 None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:48.399610 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbb4c68-b95f-4f32-8adc-4c5f795ab323 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:48.400051 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbb4c68-b95f-4f32-8adc-4c5f795ab323 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:48.402106 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbb4c68-b95f-4f32-8adc-4c5f795ab323 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:48.415828 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 213/407] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:28:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:48.416872 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb4c68-b95f-4f32-8adc-4c5f795ab323 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:48.480335 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739d7c1a-fd8f-4424-a2a1-41b8750445fb None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:48.526969 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7bb6a-e0c2-4f2d-8e8d-01ed44bf7c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:48.527840 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7bb6a-e0c2-4f2d-8e8d-01ed44bf7c3a None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:48.528445 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7bb6a-e0c2-4f2d-8e8d-01ed44bf7c3a None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:48.528894 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7bb6a-e0c2-4f2d-8e8d-01ed44bf7c3a None admin] PATH_INFO: `/v3/projects/baba6492015a48bfbca2121065b933ab` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:48.530787 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b7bb6a-e0c2-4f2d-8e8d-01ed44bf7c3a None admin] RBAC: Authorizing `identity:get_project(project_id=baba6492015a48bfbca2121065b933ab)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:48.536228 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b7bb6a-e0c2-4f2d-8e8d-01ed44bf7c3a None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:48.541526 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 195/408] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:28:48 2022] GET /identity/v3/projects/baba6492015a48bfbca2121065b933ab => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:48.541526 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7bb6a-e0c2-4f2d-8e8d-01ed44bf7c3a None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:48.544994 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb4c68-b95f-4f32-8adc-4c5f795ab323 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:48.583371 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:48.586991 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb76a5-3cb1-423b-8310-9b989ec86e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:48.587808 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb76a5-3cb1-423b-8310-9b989ec86e15 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:48.588283 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb76a5-3cb1-423b-8310-9b989ec86e15 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:48.588760 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb76a5-3cb1-423b-8310-9b989ec86e15 None admin] PATH_INFO: `/v3/groups/8503ce49c0c04a70b1c75ce4471c5c97` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:48.590113 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceb76a5-3cb1-423b-8310-9b989ec86e15 None admin] RBAC: Authorizing `identity:get_group(group_id=8503ce49c0c04a70b1c75ce4471c5c97)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:48.591831 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceb76a5-3cb1-423b-8310-9b989ec86e15 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:48.593922 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 214/409] 10.222.0.52 () {56 vars in 1180 bytes} [Mon Aug 29 22:28:48 2022] GET /identity/v3/groups/8503ce49c0c04a70b1c75ce4471c5c97 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:48.594923 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb76a5-3cb1-423b-8310-9b989ec86e15 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:48.645233 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:48.684877 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7bb6a-e0c2-4f2d-8e8d-01ed44bf7c3a None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:48.718172 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb76a5-3cb1-423b-8310-9b989ec86e15 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:48.728061 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da499cf-7637-4ef1-b012-78955c0f992c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:48.728825 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da499cf-7637-4ef1-b012-78955c0f992c None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:48.729169 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da499cf-7637-4ef1-b012-78955c0f992c None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:48.729508 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da499cf-7637-4ef1-b012-78955c0f992c None admin] PATH_INFO: `/v3/groups/0d92f1ba2d474a22a92ef4a2a01f2b9d` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:48.731083 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da499cf-7637-4ef1-b012-78955c0f992c None admin] RBAC: Authorizing `identity:get_group(group_id=0d92f1ba2d474a22a92ef4a2a01f2b9d)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:48.732915 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da499cf-7637-4ef1-b012-78955c0f992c None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:48.735335 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 196/410] 10.222.0.52 () {56 vars in 1180 bytes} [Mon Aug 29 22:28:48 2022] GET /identity/v3/groups/0d92f1ba2d474a22a92ef4a2a01f2b9d => generated 229 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:48.737077 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da499cf-7637-4ef1-b012-78955c0f992c None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:48.759417 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b702ce56-765c-46b1-8bd2-f00aae3a3645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:48.760640 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702ce56-765c-46b1-8bd2-f00aae3a3645 None admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:48.760962 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702ce56-765c-46b1-8bd2-f00aae3a3645 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:48.761241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702ce56-765c-46b1-8bd2-f00aae3a3645 None admin] PATH_INFO: `/v3/projects/baba6492015a48bfbca2121065b933ab/groups/8503ce49c0c04a70b1c75ce4471c5c97/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:48.763204 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b702ce56-765c-46b1-8bd2-f00aae3a3645 None admin] RBAC: Authorizing `identity:create_grant(project_id=baba6492015a48bfbca2121065b933ab, group_id=8503ce49c0c04a70b1c75ce4471c5c97, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:48.764642 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b702ce56-765c-46b1-8bd2-f00aae3a3645 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:48.854011 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 215/411] 10.222.0.52 () {58 vars in 1442 bytes} [Mon Aug 29 22:28:48 2022] PUT /identity/v3/projects/baba6492015a48bfbca2121065b933ab/groups/8503ce49c0c04a70b1c75ce4471c5c97/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:28:48.854841 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b702ce56-765c-46b1-8bd2-f00aae3a3645 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:48.860454 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:48.870402 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da499cf-7637-4ef1-b012-78955c0f992c None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:48.927122 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dcbd50-0011-454f-ba31-bfbb73f978cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:48.928550 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dcbd50-0011-454f-ba31-bfbb73f978cd None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:48.929636 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dcbd50-0011-454f-ba31-bfbb73f978cd None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:48.930188 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dcbd50-0011-454f-ba31-bfbb73f978cd None admin] PATH_INFO: `/v3/projects/baba6492015a48bfbca2121065b933ab` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:48.932598 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dcbd50-0011-454f-ba31-bfbb73f978cd None admin] RBAC: Authorizing `identity:get_project(project_id=baba6492015a48bfbca2121065b933ab)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:48.935799 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dcbd50-0011-454f-ba31-bfbb73f978cd None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:48.939205 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 197/412] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:28:48 2022] GET /identity/v3/projects/baba6492015a48bfbca2121065b933ab => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:48.941387 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81babb-e507-4748-b427-2dd4a3c347ec None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:48.942306 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81babb-e507-4748-b427-2dd4a3c347ec None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:48.943833 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81babb-e507-4748-b427-2dd4a3c347ec None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:48.950703 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 198/413] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:48 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:48.951883 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81babb-e507-4748-b427-2dd4a3c347ec None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:48.977237 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:48.982022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b702ce56-765c-46b1-8bd2-f00aae3a3645 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:49.031000 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae26d16d-cd61-450f-bac4-fbb9892550df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:49.031950 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae26d16d-cd61-450f-bac4-fbb9892550df None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:49.032780 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae26d16d-cd61-450f-bac4-fbb9892550df None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:49.033793 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae26d16d-cd61-450f-bac4-fbb9892550df None admin] PATH_INFO: `/v3/projects/eb4839200e8e45dfa0acd9e9d8f433bc` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:49.034981 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae26d16d-cd61-450f-bac4-fbb9892550df None admin] RBAC: Authorizing `identity:get_project(project_id=eb4839200e8e45dfa0acd9e9d8f433bc)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:49.036458 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae26d16d-cd61-450f-bac4-fbb9892550df None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:49.038482 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 216/414] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:28:48 2022] GET /identity/v3/projects/eb4839200e8e45dfa0acd9e9d8f433bc => generated 344 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:49.039739 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b25229-cab6-45b7-b270-568fd5c9ed0a None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:49.040022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b25229-cab6-45b7-b270-568fd5c9ed0a None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:49.040319 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b25229-cab6-45b7-b270-568fd5c9ed0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:49.061544 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-e7b25229-cab6-45b7-b270-568fd5c9ed0a None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:49.062698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-e7b25229-cab6-45b7-b270-568fd5c9ed0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:49.109172 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81babb-e507-4748-b427-2dd4a3c347ec None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:49.142475 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 217/415] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:49.148452 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b25229-cab6-45b7-b270-568fd5c9ed0a None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:49.165744 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958f6135-7db8-4cff-bf91-46fe65d2582d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:49.166412 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f6135-7db8-4cff-bf91-46fe65d2582d None admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:49.166689 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f6135-7db8-4cff-bf91-46fe65d2582d None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:49.166958 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f6135-7db8-4cff-bf91-46fe65d2582d None admin] PATH_INFO: `/v3/projects/baba6492015a48bfbca2121065b933ab/groups/8503ce49c0c04a70b1c75ce4471c5c97/roles/a6d1677ebe3a4355be5d67c5c78d83c7` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:49.170072 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958f6135-7db8-4cff-bf91-46fe65d2582d None admin] RBAC: Authorizing `identity:create_grant(project_id=baba6492015a48bfbca2121065b933ab, group_id=8503ce49c0c04a70b1c75ce4471c5c97, role_id=a6d1677ebe3a4355be5d67c5c78d83c7)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:49.171507 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958f6135-7db8-4cff-bf91-46fe65d2582d None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:49.270985 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:49.278032 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b25229-cab6-45b7-b270-568fd5c9ed0a None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:49.281884 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 199/416] 10.222.0.52 () {58 vars in 1442 bytes} [Mon Aug 29 22:28:48 2022] PUT /identity/v3/projects/baba6492015a48bfbca2121065b933ab/groups/8503ce49c0c04a70b1c75ce4471c5c97/roles/a6d1677ebe3a4355be5d67c5c78d83c7 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:28:49.322335 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a47e0e-1839-4e86-bb30-566c07ee2fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:49.323006 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a47e0e-1839-4e86-bb30-566c07ee2fb5 None admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:49.323334 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a47e0e-1839-4e86-bb30-566c07ee2fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:49.323615 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a47e0e-1839-4e86-bb30-566c07ee2fb5 None admin] PATH_INFO: `/v3/projects/eb4839200e8e45dfa0acd9e9d8f433bc/groups/0d92f1ba2d474a22a92ef4a2a01f2b9d/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:49.325711 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a47e0e-1839-4e86-bb30-566c07ee2fb5 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb4839200e8e45dfa0acd9e9d8f433bc, group_id=0d92f1ba2d474a22a92ef4a2a01f2b9d, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:49.327307 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a47e0e-1839-4e86-bb30-566c07ee2fb5 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:49.380468 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32a895-4519-4f65-bd27-1b78c7b1134c None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:49.380896 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32a895-4519-4f65-bd27-1b78c7b1134c None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:49.381061 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32a895-4519-4f65-bd27-1b78c7b1134c None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:49.382863 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 200/417] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:28:49.388758 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164662df-b8a4-4485-b052-c03d781cb096 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:49.389063 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164662df-b8a4-4485-b052-c03d781cb096 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:49.394680 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164662df-b8a4-4485-b052-c03d781cb096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:49.401447 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 218/418] 10.222.0.52 () {58 vars in 1442 bytes} [Mon Aug 29 22:28:49 2022] PUT /identity/v3/projects/eb4839200e8e45dfa0acd9e9d8f433bc/groups/0d92f1ba2d474a22a92ef4a2a01f2b9d/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:28:49.426862 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-164662df-b8a4-4485-b052-c03d781cb096 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:49.428260 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-164662df-b8a4-4485-b052-c03d781cb096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:49.461201 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a47e0e-1839-4e86-bb30-566c07ee2fb5 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:49.465141 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:49.535479 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 201/419] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 22:28:49.615285 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a47e0e-1839-4e86-bb30-566c07ee2fb5 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:49.653417 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10228a7-a4f3-4c75-b8ad-c372b4acb8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:49.654390 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10228a7-a4f3-4c75-b8ad-c372b4acb8c5 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:49.654864 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10228a7-a4f3-4c75-b8ad-c372b4acb8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:49.655326 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10228a7-a4f3-4c75-b8ad-c372b4acb8c5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:49.667028 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10228a7-a4f3-4c75-b8ad-c372b4acb8c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:49.674511 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-b10228a7-a4f3-4c75-b8ad-c372b4acb8c5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:28:49.674511 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:49.674511 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:49.674511 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:49.674511 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:49.674511 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:49.674511 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:49.674511 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:49.674511 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:49.674511 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:49.674511 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:49.674511 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:49.674511 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:49.674511 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:49.674511 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:49.674511 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:49.675390 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:49.675390 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:49.675390 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:49.675390 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:49.675390 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:49.675390 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:49.675390 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:49.675390 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:49.675390 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:49.675390 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:49.675390 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:49.675390 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:49.675390 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:49.675390 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:49.675390 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:49.675390 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:49.676183 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:49.676183 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:49.676183 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:49.676183 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:49.676183 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:49.676183 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:49.676183 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:49.676183 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:49.676183 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:49.676183 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:49.676183 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:28:49.676183 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:28:49.678427 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 219/420] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:28:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:28:49.686139 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164662df-b8a4-4485-b052-c03d781cb096 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:49.823643 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164662df-b8a4-4485-b052-c03d781cb096 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:49.866189 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a2113c-f275-45cf-88bd-6f8dcba2721f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:49.866801 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a2113c-f275-45cf-88bd-6f8dcba2721f None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:49.867065 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a2113c-f275-45cf-88bd-6f8dcba2721f None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:49.867340 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a2113c-f275-45cf-88bd-6f8dcba2721f None admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:49.868021 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a2113c-f275-45cf-88bd-6f8dcba2721f None admin] RBAC: Adding query filter params (name=member) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:49.868324 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a2113c-f275-45cf-88bd-6f8dcba2721f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:49.869720 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a2113c-f275-45cf-88bd-6f8dcba2721f None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:49.879464 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 202/421] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Aug 29 22:28:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:49.884629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10228a7-a4f3-4c75-b8ad-c372b4acb8c5 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:49.889533 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a2113c-f275-45cf-88bd-6f8dcba2721f None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:49.908925 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:49.930631 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:50.010628 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a2113c-f275-45cf-88bd-6f8dcba2721f None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:50.056120 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02f2474-04f4-46ad-b518-e9191f7b42ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:50.056939 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f2474-04f4-46ad-b518-e9191f7b42ac None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:50.057220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f2474-04f4-46ad-b518-e9191f7b42ac None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:50.057493 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f2474-04f4-46ad-b518-e9191f7b42ac None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:50.068201 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10228a7-a4f3-4c75-b8ad-c372b4acb8c5 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:50.085800 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02f2474-04f4-46ad-b518-e9191f7b42ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:50.091105 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-b02f2474-04f4-46ad-b518-e9191f7b42ac None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:28:50.091105 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:50.091105 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:50.091105 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:50.091105 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:50.091105 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:50.091105 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:50.091105 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:50.091105 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:50.091105 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:50.091105 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:50.091105 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:50.091105 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:50.091105 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:50.091105 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:50.091105 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:50.091966 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:50.091966 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:50.091966 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:50.091966 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:50.091966 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:50.091966 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:50.091966 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:50.091966 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:50.091966 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:50.091966 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:50.091966 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:50.091966 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:50.091966 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:50.091966 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:50.091966 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:50.091966 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:50.092782 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:50.092782 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:50.092782 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:50.092782 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:50.092782 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:50.092782 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:50.092782 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:50.092782 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:50.092782 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:50.092782 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:50.092782 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:28:50.092782 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:28:50.099338 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 203/422] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Aug 29 22:28:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:28:50.109125 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02f2474-04f4-46ad-b518-e9191f7b42ac None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:50.110951 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af4c16-4da3-4498-912f-92549c50b994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:50.111607 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af4c16-4da3-4498-912f-92549c50b994 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:50.111879 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af4c16-4da3-4498-912f-92549c50b994 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:50.112159 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af4c16-4da3-4498-912f-92549c50b994 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:50.232173 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33af4c16-4da3-4498-912f-92549c50b994 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:50.233663 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33af4c16-4da3-4498-912f-92549c50b994 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:50.262379 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:50.272245 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:50.311133 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02f2474-04f4-46ad-b518-e9191f7b42ac None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:50.353726 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 220/423] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:28:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:28:50.359798 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af4c16-4da3-4498-912f-92549c50b994 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:50.405958 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b59ea8-4ac5-47a6-922b-7120180a2cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:50.406692 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b59ea8-4ac5-47a6-922b-7120180a2cce None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:50.407037 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b59ea8-4ac5-47a6-922b-7120180a2cce None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:50.407374 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b59ea8-4ac5-47a6-922b-7120180a2cce None admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:50.408098 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b59ea8-4ac5-47a6-922b-7120180a2cce None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:50.409049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b59ea8-4ac5-47a6-922b-7120180a2cce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:50.410636 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b59ea8-4ac5-47a6-922b-7120180a2cce None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:50.435132 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 204/424] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:28:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:50.450286 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b59ea8-4ac5-47a6-922b-7120180a2cce None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:50.466598 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af4c16-4da3-4498-912f-92549c50b994 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:50.485394 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:50.505919 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e010e58d-d115-4c47-a107-acc130470305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:50.506876 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e010e58d-d115-4c47-a107-acc130470305 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:50.507357 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e010e58d-d115-4c47-a107-acc130470305 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:50.507844 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e010e58d-d115-4c47-a107-acc130470305 None admin] PATH_INFO: `/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:50.509282 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e010e58d-d115-4c47-a107-acc130470305 None admin] RBAC: Authorizing `identity:get_project(project_id=bd224ec63c3748aa90cf3e984e32fa3f)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:50.510907 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e010e58d-d115-4c47-a107-acc130470305 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:50.513066 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 221/425] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:28:50 2022] GET /identity/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:50.519319 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e010e58d-d115-4c47-a107-acc130470305 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:50.652009 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e010e58d-d115-4c47-a107-acc130470305 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:50.700416 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b59ea8-4ac5-47a6-922b-7120180a2cce None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:50.723033 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06183795-b346-4768-bf6d-ad7df2cc2b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:50.724069 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06183795-b346-4768-bf6d-ad7df2cc2b4a None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:50.725250 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06183795-b346-4768-bf6d-ad7df2cc2b4a None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:50.725716 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06183795-b346-4768-bf6d-ad7df2cc2b4a None admin] PATH_INFO: `/v3/groups/8503ce49c0c04a70b1c75ce4471c5c97` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:50.727208 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06183795-b346-4768-bf6d-ad7df2cc2b4a None admin] RBAC: Authorizing `identity:get_group(group_id=8503ce49c0c04a70b1c75ce4471c5c97)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:50.730502 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06183795-b346-4768-bf6d-ad7df2cc2b4a None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:50.732465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 222/426] 10.222.0.52 () {56 vars in 1180 bytes} [Mon Aug 29 22:28:50 2022] GET /identity/v3/groups/8503ce49c0c04a70b1c75ce4471c5c97 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:50.743800 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06183795-b346-4768-bf6d-ad7df2cc2b4a None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:50.774597 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cad3ef4-19f8-4d08-8c56-bb0c3aa760fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:50.775622 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad3ef4-19f8-4d08-8c56-bb0c3aa760fd None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:50.777079 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad3ef4-19f8-4d08-8c56-bb0c3aa760fd None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:50.778576 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad3ef4-19f8-4d08-8c56-bb0c3aa760fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:50.791538 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:50.839446 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:50.858151 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cad3ef4-19f8-4d08-8c56-bb0c3aa760fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:50.859617 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cad3ef4-19f8-4d08-8c56-bb0c3aa760fd None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:50.951493 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06183795-b346-4768-bf6d-ad7df2cc2b4a None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:50.974318 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 205/427] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:28:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:28:50.979659 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cad3ef4-19f8-4d08-8c56-bb0c3aa760fd None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:51.025980 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ef9ed-d2a5-4b9d-bfe0-d132236d48a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:51.026598 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ef9ed-d2a5-4b9d-bfe0-d132236d48a6 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:51.026880 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ef9ed-d2a5-4b9d-bfe0-d132236d48a6 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:51.027154 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ef9ed-d2a5-4b9d-bfe0-d132236d48a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:51.027885 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455ef9ed-d2a5-4b9d-bfe0-d132236d48a6 None admin] RBAC: Adding query filter params (group.id=8503ce49c0c04a70b1c75ce4471c5c97, role.id=ae06bb653e3b4990a9ddaa1a649c7e06, scope.project.id=bd224ec63c3748aa90cf3e984e32fa3f) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:51.028195 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455ef9ed-d2a5-4b9d-bfe0-d132236d48a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:51.029802 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455ef9ed-d2a5-4b9d-bfe0-d132236d48a6 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:51.037970 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 223/428] 10.222.0.52 () {56 vars in 1376 bytes} [Mon Aug 29 22:28:50 2022] GET /identity/v3/role_assignments?group.id=8503ce49c0c04a70b1c75ce4471c5c97&scope.project.id=bd224ec63c3748aa90cf3e984e32fa3f&role.id=ae06bb653e3b4990a9ddaa1a649c7e06 => generated 611 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:51.051822 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:51.122181 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cad3ef4-19f8-4d08-8c56-bb0c3aa760fd None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:51.163522 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e01060-525d-4ca6-b44f-e33a28b9f484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:51.164242 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e01060-525d-4ca6-b44f-e33a28b9f484 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:51.164528 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e01060-525d-4ca6-b44f-e33a28b9f484 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:51.164857 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e01060-525d-4ca6-b44f-e33a28b9f484 None admin] PATH_INFO: `/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:51.166035 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e01060-525d-4ca6-b44f-e33a28b9f484 None admin] RBAC: Authorizing `identity:get_project(project_id=bd224ec63c3748aa90cf3e984e32fa3f)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:51.167514 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e01060-525d-4ca6-b44f-e33a28b9f484 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:51.169609 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 206/429] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:28:50 2022] GET /identity/v3/projects/bd224ec63c3748aa90cf3e984e32fa3f => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:28:51.177921 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ef9ed-d2a5-4b9d-bfe0-d132236d48a6 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:51.301260 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331bdf8-eaa6-4371-95e8-06fa6e6a1f6f None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:51.301599 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331bdf8-eaa6-4371-95e8-06fa6e6a1f6f None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:51.301770 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331bdf8-eaa6-4371-95e8-06fa6e6a1f6f None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:51.304400 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 207/430] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:28:51.308612 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a5418-d618-4e6c-8246-3789b202932a None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:51.308939 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a5418-d618-4e6c-8246-3789b202932a None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:51.309217 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a5418-d618-4e6c-8246-3789b202932a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:51.327072 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ef9ed-d2a5-4b9d-bfe0-d132236d48a6 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:51.332470 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-ae3a5418-d618-4e6c-8246-3789b202932a None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:51.333709 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-ae3a5418-d618-4e6c-8246-3789b202932a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:51.353047 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:51.372679 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ab4244-38bc-4e4f-8fe4-1543569489b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:51.373474 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab4244-38bc-4e4f-8fe4-1543569489b7 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:51.373730 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab4244-38bc-4e4f-8fe4-1543569489b7 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:51.374008 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab4244-38bc-4e4f-8fe4-1543569489b7 None admin] PATH_INFO: `/v3/groups/8503ce49c0c04a70b1c75ce4471c5c97` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:51.375269 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ab4244-38bc-4e4f-8fe4-1543569489b7 None admin] RBAC: Authorizing `identity:get_group(group_id=8503ce49c0c04a70b1c75ce4471c5c97)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:51.376894 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ab4244-38bc-4e4f-8fe4-1543569489b7 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:51.379799 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 224/431] 10.222.0.52 () {56 vars in 1180 bytes} [Mon Aug 29 22:28:51 2022] GET /identity/v3/groups/8503ce49c0c04a70b1c75ce4471c5c97 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:51.385165 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ab4244-38bc-4e4f-8fe4-1543569489b7 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:51.436066 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 208/432] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:51.512953 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ab4244-38bc-4e4f-8fe4-1543569489b7 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:51.557006 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9bd886-8f8a-43d8-a7fd-6953fa4c8eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:51.557541 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9bd886-8f8a-43d8-a7fd-6953fa4c8eff None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:51.557820 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9bd886-8f8a-43d8-a7fd-6953fa4c8eff None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:51.558093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9bd886-8f8a-43d8-a7fd-6953fa4c8eff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:51.558889 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9bd886-8f8a-43d8-a7fd-6953fa4c8eff None admin] RBAC: Adding query filter params (group.id=8503ce49c0c04a70b1c75ce4471c5c97, role.id=a6d1677ebe3a4355be5d67c5c78d83c7, scope.project.id=bd224ec63c3748aa90cf3e984e32fa3f) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:51.559198 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9bd886-8f8a-43d8-a7fd-6953fa4c8eff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:51.560821 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9bd886-8f8a-43d8-a7fd-6953fa4c8eff None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:51.567649 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 225/433] 10.222.0.52 () {56 vars in 1376 bytes} [Mon Aug 29 22:28:51 2022] GET /identity/v3/role_assignments?group.id=8503ce49c0c04a70b1c75ce4471c5c97&scope.project.id=bd224ec63c3748aa90cf3e984e32fa3f&role.id=a6d1677ebe3a4355be5d67c5c78d83c7 => generated 611 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:28:51.663322 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a93a1b-71a9-4aa6-9bad-388b5992af85 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:51.663674 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a93a1b-71a9-4aa6-9bad-388b5992af85 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:51.663873 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a93a1b-71a9-4aa6-9bad-388b5992af85 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:51.665561 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 209/434] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:51.676789 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f3fae-7549-4df7-b2dc-edb275953cb3 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:51.677096 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f3fae-7549-4df7-b2dc-edb275953cb3 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:51.677337 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f3fae-7549-4df7-b2dc-edb275953cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:51.700929 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-851f3fae-7549-4df7-b2dc-edb275953cb3 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:51.702160 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-851f3fae-7549-4df7-b2dc-edb275953cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:51.753462 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a93a1b-71a9-4aa6-9bad-388b5992af85 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:51.758307 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:51.791938 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 226/435] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:51.793693 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea1c40-e63b-4e79-b370-8345264f9b9a None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:51.793977 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea1c40-e63b-4e79-b370-8345264f9b9a None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:51.794252 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea1c40-e63b-4e79-b370-8345264f9b9a None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:51.796046 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 227/436] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:51.803823 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ff088-3a1c-48b9-bb3c-1d30b3b0131a None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:51.804119 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ff088-3a1c-48b9-bb3c-1d30b3b0131a None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:51.804377 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ff088-3a1c-48b9-bb3c-1d30b3b0131a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:51.808425 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:51.826103 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-d16ff088-3a1c-48b9-bb3c-1d30b3b0131a None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:51.827310 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-d16ff088-3a1c-48b9-bb3c-1d30b3b0131a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:51.902431 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a93a1b-71a9-4aa6-9bad-388b5992af85 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:51.917308 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 228/437] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 22:28:51.949763 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa76ad7-fda7-42bd-88df-111fea4f1325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:51.950395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa76ad7-fda7-42bd-88df-111fea4f1325 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:51.950665 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa76ad7-fda7-42bd-88df-111fea4f1325 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:51.950930 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa76ad7-fda7-42bd-88df-111fea4f1325 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:51.963978 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa76ad7-fda7-42bd-88df-111fea4f1325 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:51.969081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-1fa76ad7-fda7-42bd-88df-111fea4f1325 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:28:51.969081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:51.969081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:51.969081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:51.969081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:51.969081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:51.969081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:51.969081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:51.969081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:51.969081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:51.969081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:51.969081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:51.969081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:51.969081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:51.969081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:51.969081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:51.969939 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:51.969939 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:51.969939 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:51.969939 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:51.969939 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:51.969939 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:51.969939 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:51.969939 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:51.969939 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:51.969939 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:51.969939 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:51.969939 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:51.969939 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:51.969939 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:51.969939 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:51.969939 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:51.970901 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:51.970901 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:51.970901 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:51.970901 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:51.970901 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:51.970901 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:51.970901 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:51.970901 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:51.970901 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:51.970901 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:51.970901 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:28:51.970901 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:28:51.973190 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 210/438] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:28:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:51.980143 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ff088-3a1c-48b9-bb3c-1d30b3b0131a None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:52.081935 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa76ad7-fda7-42bd-88df-111fea4f1325 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:52.097109 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ff088-3a1c-48b9-bb3c-1d30b3b0131a None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:52.108053 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:52.140663 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfe460e-2469-45b4-b887-2f069c83a691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:52.141314 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe460e-2469-45b4-b887-2f069c83a691 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:52.141589 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe460e-2469-45b4-b887-2f069c83a691 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:52.141855 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe460e-2469-45b4-b887-2f069c83a691 None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:52.142541 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfe460e-2469-45b4-b887-2f069c83a691 None admin] RBAC: Adding query filter params (name=member) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:52.143441 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfe460e-2469-45b4-b887-2f069c83a691 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:52.146844 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfe460e-2469-45b4-b887-2f069c83a691 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:52.157144 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 229/439] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Aug 29 22:28:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:52.162984 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfe460e-2469-45b4-b887-2f069c83a691 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:52.204807 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:52.211473 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa76ad7-fda7-42bd-88df-111fea4f1325 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:52.245398 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc68d79b-eb23-4af2-943b-6a718eb96528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:52.246021 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68d79b-eb23-4af2-943b-6a718eb96528 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:52.246315 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68d79b-eb23-4af2-943b-6a718eb96528 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:52.246584 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68d79b-eb23-4af2-943b-6a718eb96528 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:52.257166 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc68d79b-eb23-4af2-943b-6a718eb96528 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:52.261939 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-dc68d79b-eb23-4af2-943b-6a718eb96528 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:28:52.261939 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:52.261939 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:52.261939 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:52.261939 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:52.261939 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:52.261939 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:52.261939 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:52.261939 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:52.261939 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:52.261939 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:52.261939 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:52.261939 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:52.261939 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:52.261939 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:52.261939 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:52.262852 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:52.262852 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:52.262852 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:52.262852 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:52.262852 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:52.262852 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:52.262852 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:52.262852 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:52.262852 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:52.262852 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:52.262852 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:52.262852 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:52.262852 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:52.262852 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:52.262852 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:52.262852 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:52.263764 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:52.263764 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:52.263764 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:52.263764 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:52.263764 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:52.263764 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:52.263764 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:52.263764 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:52.263764 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:52.263764 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:52.263764 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:28:52.263764 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:28:52.263764 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 211/440] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Aug 29 22:28:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:28:52.263764 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc68d79b-eb23-4af2-943b-6a718eb96528 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:52.273740 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfe460e-2469-45b4-b887-2f069c83a691 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:52.307387 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a688f-c060-4252-a4d3-6040cd437393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:52.307991 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a688f-c060-4252-a4d3-6040cd437393 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:52.308277 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a688f-c060-4252-a4d3-6040cd437393 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:52.308553 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a688f-c060-4252-a4d3-6040cd437393 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:52.348549 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:52.365997 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc68d79b-eb23-4af2-943b-6a718eb96528 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:52.376032 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629a688f-c060-4252-a4d3-6040cd437393 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:52.378327 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629a688f-c060-4252-a4d3-6040cd437393 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:52.401050 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a22fea7-9bf0-4634-87c3-797b86dd3a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:52.401665 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22fea7-9bf0-4634-87c3-797b86dd3a7f None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:52.401941 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22fea7-9bf0-4634-87c3-797b86dd3a7f None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:52.402210 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22fea7-9bf0-4634-87c3-797b86dd3a7f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:52.412290 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a22fea7-9bf0-4634-87c3-797b86dd3a7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:52.417530 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-1a22fea7-9bf0-4634-87c3-797b86dd3a7f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:28:52.417530 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:52.417530 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:52.417530 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:52.417530 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:52.417530 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:52.417530 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:52.417530 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:52.417530 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:52.417530 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:52.417530 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:52.417530 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:52.417530 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:52.417530 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:52.417530 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:52.417530 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:52.418625 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:52.418625 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:52.418625 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:52.418625 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:52.418625 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:52.418625 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:52.418625 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:52.418625 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:52.418625 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:52.418625 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:52.418625 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:52.418625 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:52.418625 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:52.418625 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:52.418625 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:52.418625 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:52.419449 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:52.419449 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:52.419449 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:52.419449 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:52.419449 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:52.419449 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:52.419449 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:52.419449 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:52.419449 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:52.419449 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:52.419449 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:28:52.419449 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:28:52.419449 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 212/441] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Aug 29 22:28:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:52.420206 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a22fea7-9bf0-4634-87c3-797b86dd3a7f None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:52.438014 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:52.485763 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 230/442] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:28:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:28:52.485763 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a688f-c060-4252-a4d3-6040cd437393 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:52.521470 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a22fea7-9bf0-4634-87c3-797b86dd3a7f None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:52.552554 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:52.558402 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64a7aaa-814a-4c9d-a01b-4a7c73fb7978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:52.559017 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64a7aaa-814a-4c9d-a01b-4a7c73fb7978 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:52.559294 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64a7aaa-814a-4c9d-a01b-4a7c73fb7978 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:52.559566 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64a7aaa-814a-4c9d-a01b-4a7c73fb7978 None admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:52.560277 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64a7aaa-814a-4c9d-a01b-4a7c73fb7978 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:52.560578 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64a7aaa-814a-4c9d-a01b-4a7c73fb7978 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:52.561983 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64a7aaa-814a-4c9d-a01b-4a7c73fb7978 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:52.570908 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 213/443] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:28:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:52.571692 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64a7aaa-814a-4c9d-a01b-4a7c73fb7978 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:52.584009 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a688f-c060-4252-a4d3-6040cd437393 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:52.619079 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e7e52e-5559-4785-99ad-3e58ea3c14a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:52.619602 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e7e52e-5559-4785-99ad-3e58ea3c14a4 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:52.619879 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e7e52e-5559-4785-99ad-3e58ea3c14a4 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:52.620149 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e7e52e-5559-4785-99ad-3e58ea3c14a4 None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:52.620824 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e7e52e-5559-4785-99ad-3e58ea3c14a4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:52.621128 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e7e52e-5559-4785-99ad-3e58ea3c14a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:52.622475 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e7e52e-5559-4785-99ad-3e58ea3c14a4 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:52.630983 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 231/444] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:28:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:52.631766 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e7e52e-5559-4785-99ad-3e58ea3c14a4 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:52.659220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:52.681392 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64a7aaa-814a-4c9d-a01b-4a7c73fb7978 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:52.722541 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a593167-a949-40ef-8ea5-aedb73a6fbe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:52.723163 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a593167-a949-40ef-8ea5-aedb73a6fbe0 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:52.723448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a593167-a949-40ef-8ea5-aedb73a6fbe0 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:52.723721 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a593167-a949-40ef-8ea5-aedb73a6fbe0 None admin] PATH_INFO: `/v3/projects/baba6492015a48bfbca2121065b933ab` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:52.724868 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a593167-a949-40ef-8ea5-aedb73a6fbe0 None admin] RBAC: Authorizing `identity:get_project(project_id=baba6492015a48bfbca2121065b933ab)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:52.726306 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a593167-a949-40ef-8ea5-aedb73a6fbe0 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:52.727920 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 214/445] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:28:52 2022] GET /identity/v3/projects/baba6492015a48bfbca2121065b933ab => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:52.728676 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a593167-a949-40ef-8ea5-aedb73a6fbe0 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:52.744970 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e7e52e-5559-4785-99ad-3e58ea3c14a4 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:52.775012 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:52.780384 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af28039-d029-46dd-b441-905de4e87088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:52.781017 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af28039-d029-46dd-b441-905de4e87088 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:52.781296 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af28039-d029-46dd-b441-905de4e87088 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:52.781566 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af28039-d029-46dd-b441-905de4e87088 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:52.828978 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a593167-a949-40ef-8ea5-aedb73a6fbe0 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:52.851018 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af28039-d029-46dd-b441-905de4e87088 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:52.852488 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af28039-d029-46dd-b441-905de4e87088 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:52.862738 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a87ad3-68f6-41f8-89db-036bfc2a2f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:52.863333 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a87ad3-68f6-41f8-89db-036bfc2a2f82 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:52.863614 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a87ad3-68f6-41f8-89db-036bfc2a2f82 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:52.863887 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a87ad3-68f6-41f8-89db-036bfc2a2f82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:52.879059 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:52.933297 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a87ad3-68f6-41f8-89db-036bfc2a2f82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:52.934749 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a87ad3-68f6-41f8-89db-036bfc2a2f82 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:52.953042 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 232/446] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:28:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:28:52.953878 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af28039-d029-46dd-b441-905de4e87088 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:52.984104 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:53.037386 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 215/447] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:28:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:28:53.038068 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a87ad3-68f6-41f8-89db-036bfc2a2f82 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:53.058235 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af28039-d029-46dd-b441-905de4e87088 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:53.088943 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:53.095284 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73323bda-addc-4c49-8306-0cabda8e7073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:53.096451 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73323bda-addc-4c49-8306-0cabda8e7073 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:53.096808 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73323bda-addc-4c49-8306-0cabda8e7073 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:53.097091 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73323bda-addc-4c49-8306-0cabda8e7073 None admin] PATH_INFO: `/v3/groups/8503ce49c0c04a70b1c75ce4471c5c97` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:53.100933 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73323bda-addc-4c49-8306-0cabda8e7073 None admin] RBAC: Authorizing `identity:get_group(group_id=8503ce49c0c04a70b1c75ce4471c5c97)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:53.100933 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73323bda-addc-4c49-8306-0cabda8e7073 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:53.105986 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 233/448] 10.222.0.52 () {56 vars in 1180 bytes} [Mon Aug 29 22:28:52 2022] GET /identity/v3/groups/8503ce49c0c04a70b1c75ce4471c5c97 => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:53.105986 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73323bda-addc-4c49-8306-0cabda8e7073 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:53.150570 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a87ad3-68f6-41f8-89db-036bfc2a2f82 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:53.185223 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac17cf4-a9f7-4a69-8e2f-7f1629e9ce61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:53.186090 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac17cf4-a9f7-4a69-8e2f-7f1629e9ce61 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:53.186366 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac17cf4-a9f7-4a69-8e2f-7f1629e9ce61 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:53.186636 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac17cf4-a9f7-4a69-8e2f-7f1629e9ce61 None admin] PATH_INFO: `/v3/projects/eb4839200e8e45dfa0acd9e9d8f433bc` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:53.187743 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac17cf4-a9f7-4a69-8e2f-7f1629e9ce61 None admin] RBAC: Authorizing `identity:get_project(project_id=eb4839200e8e45dfa0acd9e9d8f433bc)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:53.189239 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac17cf4-a9f7-4a69-8e2f-7f1629e9ce61 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:53.191358 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 216/449] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:28:53 2022] GET /identity/v3/projects/eb4839200e8e45dfa0acd9e9d8f433bc => generated 344 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:53.193381 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac17cf4-a9f7-4a69-8e2f-7f1629e9ce61 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:53.198843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:53.201816 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73323bda-addc-4c49-8306-0cabda8e7073 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:53.237690 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674b7ac8-f6d7-4835-8bd5-dc8490e85df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:53.238324 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b7ac8-f6d7-4835-8bd5-dc8490e85df1 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:53.238602 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b7ac8-f6d7-4835-8bd5-dc8490e85df1 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:53.238875 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b7ac8-f6d7-4835-8bd5-dc8490e85df1 None admin] PATH_INFO: `/v3/projects/baba6492015a48bfbca2121065b933ab` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:53.239988 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674b7ac8-f6d7-4835-8bd5-dc8490e85df1 None admin] RBAC: Authorizing `identity:get_project(project_id=baba6492015a48bfbca2121065b933ab)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:53.241458 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674b7ac8-f6d7-4835-8bd5-dc8490e85df1 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:53.243069 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 234/450] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:28:53 2022] GET /identity/v3/projects/baba6492015a48bfbca2121065b933ab => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:53.243821 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674b7ac8-f6d7-4835-8bd5-dc8490e85df1 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:53.300218 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac17cf4-a9f7-4a69-8e2f-7f1629e9ce61 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:53.313974 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:53.335175 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3949092a-a521-4d8e-b32c-d04707b2e38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:53.335770 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949092a-a521-4d8e-b32c-d04707b2e38c None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:53.336049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949092a-a521-4d8e-b32c-d04707b2e38c None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:53.336326 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949092a-a521-4d8e-b32c-d04707b2e38c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:53.337122 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3949092a-a521-4d8e-b32c-d04707b2e38c None admin] RBAC: Adding query filter params (group.id=8503ce49c0c04a70b1c75ce4471c5c97, role.id=ae06bb653e3b4990a9ddaa1a649c7e06, scope.project.id=baba6492015a48bfbca2121065b933ab) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:53.337432 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3949092a-a521-4d8e-b32c-d04707b2e38c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:53.341221 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3949092a-a521-4d8e-b32c-d04707b2e38c None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:53.344535 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 217/451] 10.222.0.52 () {56 vars in 1376 bytes} [Mon Aug 29 22:28:53 2022] GET /identity/v3/role_assignments?group.id=8503ce49c0c04a70b1c75ce4471c5c97&scope.project.id=baba6492015a48bfbca2121065b933ab&role.id=ae06bb653e3b4990a9ddaa1a649c7e06 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:53.345414 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3949092a-a521-4d8e-b32c-d04707b2e38c None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:53.349238 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674b7ac8-f6d7-4835-8bd5-dc8490e85df1 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:53.390812 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb00900-c258-4e53-af40-84c4bbedc18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:53.391588 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb00900-c258-4e53-af40-84c4bbedc18d None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:53.391872 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb00900-c258-4e53-af40-84c4bbedc18d None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:53.392142 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb00900-c258-4e53-af40-84c4bbedc18d None admin] PATH_INFO: `/v3/groups/0d92f1ba2d474a22a92ef4a2a01f2b9d` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:53.394132 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb00900-c258-4e53-af40-84c4bbedc18d None admin] RBAC: Authorizing `identity:get_group(group_id=0d92f1ba2d474a22a92ef4a2a01f2b9d)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:53.395771 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb00900-c258-4e53-af40-84c4bbedc18d None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:53.397844 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 235/452] 10.222.0.52 () {56 vars in 1180 bytes} [Mon Aug 29 22:28:53 2022] GET /identity/v3/groups/0d92f1ba2d474a22a92ef4a2a01f2b9d => generated 229 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:53.403778 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb00900-c258-4e53-af40-84c4bbedc18d None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:53.444113 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:53.476081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3949092a-a521-4d8e-b32c-d04707b2e38c None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:53.520036 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b52233-06bf-4dd3-a25f-ebd46e1f43f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:53.520672 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b52233-06bf-4dd3-a25f-ebd46e1f43f2 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:53.520984 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b52233-06bf-4dd3-a25f-ebd46e1f43f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:53.521259 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b52233-06bf-4dd3-a25f-ebd46e1f43f2 None admin] PATH_INFO: `/v3/groups/8503ce49c0c04a70b1c75ce4471c5c97` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:53.522377 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b52233-06bf-4dd3-a25f-ebd46e1f43f2 None admin] RBAC: Authorizing `identity:get_group(group_id=8503ce49c0c04a70b1c75ce4471c5c97)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:53.523819 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b52233-06bf-4dd3-a25f-ebd46e1f43f2 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:53.526492 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 218/453] 10.222.0.52 () {56 vars in 1180 bytes} [Mon Aug 29 22:28:53 2022] GET /identity/v3/groups/8503ce49c0c04a70b1c75ce4471c5c97 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:53.526606 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb00900-c258-4e53-af40-84c4bbedc18d None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:53.531274 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b52233-06bf-4dd3-a25f-ebd46e1f43f2 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:53.578930 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0977e2a-2a54-435a-ab27-c055f1ed869a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:53.579548 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0977e2a-2a54-435a-ab27-c055f1ed869a None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:53.579829 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0977e2a-2a54-435a-ab27-c055f1ed869a None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:53.580178 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0977e2a-2a54-435a-ab27-c055f1ed869a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:53.581161 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0977e2a-2a54-435a-ab27-c055f1ed869a None admin] RBAC: Adding query filter params (group.id=0d92f1ba2d474a22a92ef4a2a01f2b9d, role.id=882f35bd847a406eb2222aeafc517a46, scope.project.id=eb4839200e8e45dfa0acd9e9d8f433bc) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:53.581475 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0977e2a-2a54-435a-ab27-c055f1ed869a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:53.583125 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0977e2a-2a54-435a-ab27-c055f1ed869a None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:53.589404 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:53.591999 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 236/454] 10.222.0.52 () {56 vars in 1376 bytes} [Mon Aug 29 22:28:53 2022] GET /identity/v3/role_assignments?group.id=0d92f1ba2d474a22a92ef4a2a01f2b9d&scope.project.id=eb4839200e8e45dfa0acd9e9d8f433bc&role.id=882f35bd847a406eb2222aeafc517a46 => generated 611 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:28:53.660159 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b52233-06bf-4dd3-a25f-ebd46e1f43f2 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:53.699152 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794a2e9d-6649-4531-bb6a-72d14ea38ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:53.699763 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a2e9d-6649-4531-bb6a-72d14ea38ea0 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:53.700045 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a2e9d-6649-4531-bb6a-72d14ea38ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:53.700319 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a2e9d-6649-4531-bb6a-72d14ea38ea0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:53.701094 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794a2e9d-6649-4531-bb6a-72d14ea38ea0 None admin] RBAC: Adding query filter params (group.id=8503ce49c0c04a70b1c75ce4471c5c97, role.id=a6d1677ebe3a4355be5d67c5c78d83c7, scope.project.id=baba6492015a48bfbca2121065b933ab) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:53.701403 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794a2e9d-6649-4531-bb6a-72d14ea38ea0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:53.702871 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794a2e9d-6649-4531-bb6a-72d14ea38ea0 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:53.709867 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 219/455] 10.222.0.52 () {56 vars in 1376 bytes} [Mon Aug 29 22:28:53 2022] GET /identity/v3/role_assignments?group.id=8503ce49c0c04a70b1c75ce4471c5c97&scope.project.id=baba6492015a48bfbca2121065b933ab&role.id=a6d1677ebe3a4355be5d67c5c78d83c7 => generated 611 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:28:56.374639 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8665bb-5af4-4d18-9e43-ab89e589663a None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:56.375482 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8665bb-5af4-4d18-9e43-ab89e589663a None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:56.375984 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8665bb-5af4-4d18-9e43-ab89e589663a None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:56.377782 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 237/456] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:56.383452 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85149c5f-c892-44a9-be99-53be2e402339 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:56.384244 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85149c5f-c892-44a9-be99-53be2e402339 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:56.384659 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85149c5f-c892-44a9-be99-53be2e402339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:56.397871 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900b30d-661d-4b6c-9ff8-c37a72b5730d None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:56.398495 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900b30d-661d-4b6c-9ff8-c37a72b5730d None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:56.398921 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900b30d-661d-4b6c-9ff8-c37a72b5730d None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:56.400367 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 238/457] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:28:56.405111 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4d0d1-7846-45e4-bc01-7d7344f3cf29 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:56.405572 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4d0d1-7846-45e4-bc01-7d7344f3cf29 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:56.405983 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4d0d1-7846-45e4-bc01-7d7344f3cf29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:56.432971 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-85149c5f-c892-44a9-be99-53be2e402339 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:56.434571 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-85149c5f-c892-44a9-be99-53be2e402339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:56.435615 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-58b4d0d1-7846-45e4-bc01-7d7344f3cf29 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:56.438954 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-58b4d0d1-7846-45e4-bc01-7d7344f3cf29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:56.574985 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 220/458] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:56.582130 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 239/459] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 22:28:56.628194 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c1d2b-7888-4f43-8b99-9c5d78acc427 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:56.628730 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c1d2b-7888-4f43-8b99-9c5d78acc427 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:56.629149 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c1d2b-7888-4f43-8b99-9c5d78acc427 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:56.630641 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 221/460] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:56.635872 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0daa61-326b-4fd4-bf10-287be2105fe0 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:56.636462 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0daa61-326b-4fd4-bf10-287be2105fe0 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:56.636921 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0daa61-326b-4fd4-bf10-287be2105fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:56.661418 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-fe0daa61-326b-4fd4-bf10-287be2105fe0 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:56.662840 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-fe0daa61-326b-4fd4-bf10-287be2105fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:56.689267 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:56.734769 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 240/461] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:56.876964 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762c1d2b-7888-4f43-8b99-9c5d78acc427 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:56.888013 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0daa61-326b-4fd4-bf10-287be2105fe0 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:56.965515 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:57.047928 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762c1d2b-7888-4f43-8b99-9c5d78acc427 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:57.056520 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0daa61-326b-4fd4-bf10-287be2105fe0 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:57.089343 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24191a92-46b0-4f0c-86c7-3e0df4df4fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:57.090386 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24191a92-46b0-4f0c-86c7-3e0df4df4fb3 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:57.090803 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24191a92-46b0-4f0c-86c7-3e0df4df4fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:57.091396 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24191a92-46b0-4f0c-86c7-3e0df4df4fb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:57.095193 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7dab77-abaf-46e0-b349-bd2e36498a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:57.096431 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7dab77-abaf-46e0-b349-bd2e36498a5c None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:57.096824 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7dab77-abaf-46e0-b349-bd2e36498a5c None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:57.097194 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7dab77-abaf-46e0-b349-bd2e36498a5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:57.104996 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24191a92-46b0-4f0c-86c7-3e0df4df4fb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:57.107111 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24191a92-46b0-4f0c-86c7-3e0df4df4fb3 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:57.107899 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7dab77-abaf-46e0-b349-bd2e36498a5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:57.109371 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7dab77-abaf-46e0-b349-bd2e36498a5c None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:57.125362 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-24191a92-46b0-4f0c-86c7-3e0df4df4fb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:28:57.125362 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:57.125362 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:57.125362 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:57.125362 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:57.125362 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:57.125362 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:57.125362 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:57.125362 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:57.125362 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:57.125362 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:57.125362 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:57.125362 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:28:57.125362 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:28:57.125362 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:28:57.125362 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:28:57.128161 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:57.128161 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:57.128161 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:57.128161 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:57.128161 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:57.128161 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:57.128161 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:57.128161 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:57.128161 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:57.128161 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:57.128161 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:57.128161 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:57.128161 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:57.128161 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:57.128161 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:57.128161 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:57.128161 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:28:57.129072 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:28:57.129072 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:28:57.129072 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:28:57.129072 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 222/462] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:28:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:57.129072 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d8ff6-04ff-4cef-aded-4371fbc0ab34 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:57.129072 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d8ff6-04ff-4cef-aded-4371fbc0ab34 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:57.129396 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d8ff6-04ff-4cef-aded-4371fbc0ab34 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:57.132106 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 223/463] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:57.132106 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d8ff6-04ff-4cef-aded-4371fbc0ab34 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:57.133182 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-8a7dab77-abaf-46e0-b349-bd2e36498a5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:28:57.133182 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:57.133182 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:57.133182 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:57.133182 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:57.133182 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:57.133182 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:57.133182 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:57.133182 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:57.133182 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:57.133182 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:57.133182 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:57.133182 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:28:57.133182 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:28:57.133182 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:28:57.133182 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:28:57.134022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:57.134022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:57.134022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:57.134022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:57.134022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:57.134022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:57.134022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:57.134022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:57.134022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:57.134022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:57.134022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:57.134022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:57.134022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:57.134022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:57.134022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:57.134022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:57.134022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:28:57.134922 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:28:57.134922 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:28:57.134922 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:28:57.136686 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 241/464] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:28:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:57.137883 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7dab77-abaf-46e0-b349-bd2e36498a5c None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:57.157888 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:57.244588 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:57.257866 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d8ff6-04ff-4cef-aded-4371fbc0ab34 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:57.262147 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7dab77-abaf-46e0-b349-bd2e36498a5c None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:57.297565 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dd11e2-5534-48fb-b95c-cc5c3171215f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:57.298194 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd11e2-5534-48fb-b95c-cc5c3171215f None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:57.299016 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd11e2-5534-48fb-b95c-cc5c3171215f None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:57.299016 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd11e2-5534-48fb-b95c-cc5c3171215f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:57.301946 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851c1d82-7aa3-4056-9b19-531424820d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:57.302553 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c1d82-7aa3-4056-9b19-531424820d7a None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:57.302830 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c1d82-7aa3-4056-9b19-531424820d7a None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:57.303162 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c1d82-7aa3-4056-9b19-531424820d7a None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:57.303796 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851c1d82-7aa3-4056-9b19-531424820d7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:57.304145 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851c1d82-7aa3-4056-9b19-531424820d7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:57.305544 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851c1d82-7aa3-4056-9b19-531424820d7a None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:57.309752 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dd11e2-5534-48fb-b95c-cc5c3171215f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:57.311204 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dd11e2-5534-48fb-b95c-cc5c3171215f None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:57.328843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 242/465] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:28:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:57.328843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de6dbb-82ab-4dfc-a1b3-338c60b8fd3e None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:57.328843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de6dbb-82ab-4dfc-a1b3-338c60b8fd3e None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:57.328843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de6dbb-82ab-4dfc-a1b3-338c60b8fd3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:57.329216 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-44dd11e2-5534-48fb-b95c-cc5c3171215f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:28:57.329216 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:57.329216 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:57.329216 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:57.329216 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:57.329216 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:57.329216 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:57.329216 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:57.329216 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:57.329216 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:57.329216 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:57.329216 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:57.329216 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:28:57.329216 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:28:57.329216 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:28:57.329216 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:28:57.330081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:57.330081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:57.330081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:57.330081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:57.330081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:57.330081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:57.330081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:57.330081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:57.330081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:57.330081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:57.330081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:57.330081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:57.330081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:57.330081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:57.330081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:57.330081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:57.330081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:28:57.331691 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:28:57.331691 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:28:57.331691 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:28:57.331691 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 224/466] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:28:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:57.331691 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dd11e2-5534-48fb-b95c-cc5c3171215f None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:57.350510 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-26de6dbb-82ab-4dfc-a1b3-338c60b8fd3e None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:57.351698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-26de6dbb-82ab-4dfc-a1b3-338c60b8fd3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:57.433290 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 243/467] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:28:57.434076 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de6dbb-82ab-4dfc-a1b3-338c60b8fd3e None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:57.440799 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dd11e2-5534-48fb-b95c-cc5c3171215f None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:57.441410 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:57.486464 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9b425b-d0ba-4dac-b65d-e3854300234f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:57.487598 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b425b-d0ba-4dac-b65d-e3854300234f None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:57.487885 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b425b-d0ba-4dac-b65d-e3854300234f None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:57.488165 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b425b-d0ba-4dac-b65d-e3854300234f None admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:57.488873 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9b425b-d0ba-4dac-b65d-e3854300234f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:57.489184 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9b425b-d0ba-4dac-b65d-e3854300234f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:57.491092 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9b425b-d0ba-4dac-b65d-e3854300234f None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:57.510145 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 225/468] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:28:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:57.511370 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9b425b-d0ba-4dac-b65d-e3854300234f None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:57.516302 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:57.570138 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de6dbb-82ab-4dfc-a1b3-338c60b8fd3e None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:57.605417 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573c59e-2cca-45be-90c1-50b4059421aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:57.606281 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573c59e-2cca-45be-90c1-50b4059421aa None admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:57.606696 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573c59e-2cca-45be-90c1-50b4059421aa None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:57.607100 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573c59e-2cca-45be-90c1-50b4059421aa None admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:57.607871 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e573c59e-2cca-45be-90c1-50b4059421aa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:57.609475 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e573c59e-2cca-45be-90c1-50b4059421aa None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:57.644850 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9b425b-d0ba-4dac-b65d-e3854300234f None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:57.682044 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ec0226-f802-4344-883a-3c5a928ed38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:57.682648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec0226-f802-4344-883a-3c5a928ed38e None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:57.682931 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec0226-f802-4344-883a-3c5a928ed38e None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:57.683205 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec0226-f802-4344-883a-3c5a928ed38e None admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:57.683897 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ec0226-f802-4344-883a-3c5a928ed38e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:57.684152 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ec0226-f802-4344-883a-3c5a928ed38e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:57.685659 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ec0226-f802-4344-883a-3c5a928ed38e None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:57.702737 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 226/469] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:28:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:57.702737 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ec0226-f802-4344-883a-3c5a928ed38e None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:57.717868 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 244/470] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Aug 29 22:28:57 2022] POST /identity/v3/users => generated 248 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:28:57.718236 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573c59e-2cca-45be-90c1-50b4059421aa None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:57.792922 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:57.808490 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:57.838805 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ec0226-f802-4344-883a-3c5a928ed38e None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:57.860659 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573c59e-2cca-45be-90c1-50b4059421aa None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:57.876069 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc9a6c-d448-4fed-99a8-b10b23d69c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:57.877031 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc9a6c-d448-4fed-99a8-b10b23d69c2a None admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:57.877031 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc9a6c-d448-4fed-99a8-b10b23d69c2a None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:57.877803 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc9a6c-d448-4fed-99a8-b10b23d69c2a None admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:57.879265 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc9a6c-d448-4fed-99a8-b10b23d69c2a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:57.880007 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc9a6c-d448-4fed-99a8-b10b23d69c2a None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:57.903150 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956d054c-c67d-42f7-a6bb-4fa3969389e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:57.903746 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d054c-c67d-42f7-a6bb-4fa3969389e5 None admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:57.904028 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d054c-c67d-42f7-a6bb-4fa3969389e5 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:57.904290 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d054c-c67d-42f7-a6bb-4fa3969389e5 None admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:57.905088 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956d054c-c67d-42f7-a6bb-4fa3969389e5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:57.906534 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956d054c-c67d-42f7-a6bb-4fa3969389e5 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:57.967821 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 227/471] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Aug 29 22:28:57 2022] POST /identity/v3/users => generated 246 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:28:57.968669 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc9a6c-d448-4fed-99a8-b10b23d69c2a None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:57.996929 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 245/472] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Aug 29 22:28:57 2022] POST /identity/v3/users => generated 248 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:28:58.001930 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05007f39-ca38-499c-848a-14abdc1e4025 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:58.001930 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05007f39-ca38-499c-848a-14abdc1e4025 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:58.001930 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05007f39-ca38-499c-848a-14abdc1e4025 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:58.006933 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 246/473] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:57 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:58.008409 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2033ce-1cae-478b-8f87-523add74f877 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:58.009915 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2033ce-1cae-478b-8f87-523add74f877 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:58.010216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2033ce-1cae-478b-8f87-523add74f877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:58.048083 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-5e2033ce-1cae-478b-8f87-523add74f877 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:58.056211 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-5e2033ce-1cae-478b-8f87-523add74f877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:58.174595 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc9a6c-d448-4fed-99a8-b10b23d69c2a None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:58.178068 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 247/474] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 22:28:58.201325 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:58.216580 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e894c8f-6c55-46e1-8679-5f708956c33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:58.218798 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e894c8f-6c55-46e1-8679-5f708956c33d None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:58.218798 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e894c8f-6c55-46e1-8679-5f708956c33d None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:58.218798 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e894c8f-6c55-46e1-8679-5f708956c33d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:58.229368 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e894c8f-6c55-46e1-8679-5f708956c33d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:58.230830 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e894c8f-6c55-46e1-8679-5f708956c33d None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:58.245827 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-9e894c8f-6c55-46e1-8679-5f708956c33d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:28:58.245827 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:58.245827 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:58.245827 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:58.245827 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:58.245827 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:58.245827 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:58.245827 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:58.245827 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:58.245827 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:58.245827 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:58.245827 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:58.245827 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:28:58.245827 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:28:58.245827 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:28:58.245827 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:28:58.246780 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:58.246780 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:58.246780 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:58.246780 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:58.246780 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:58.246780 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:58.246780 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:58.246780 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:58.246780 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:58.246780 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:58.246780 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:58.246780 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:58.246780 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:58.246780 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:58.246780 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:58.246780 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:58.246780 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:28:58.247780 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:28:58.247780 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:28:58.247780 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:28:58.247780 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 228/475] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:28:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:58.255575 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2033ce-1cae-478b-8f87-523add74f877 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:58.309600 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:58.375689 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2033ce-1cae-478b-8f87-523add74f877 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:58.416831 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000ea4f9-8b0c-4b12-974a-0706e67a2f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:58.417447 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000ea4f9-8b0c-4b12-974a-0706e67a2f15 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:58.417732 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000ea4f9-8b0c-4b12-974a-0706e67a2f15 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:58.418004 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000ea4f9-8b0c-4b12-974a-0706e67a2f15 None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:58.418642 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000ea4f9-8b0c-4b12-974a-0706e67a2f15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:58.418944 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000ea4f9-8b0c-4b12-974a-0706e67a2f15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:58.420310 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000ea4f9-8b0c-4b12-974a-0706e67a2f15 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:58.433556 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 248/476] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:28:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:58.442492 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e894c8f-6c55-46e1-8679-5f708956c33d None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:58.473624 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000ea4f9-8b0c-4b12-974a-0706e67a2f15 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:58.564074 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e894c8f-6c55-46e1-8679-5f708956c33d None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:58.602183 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1330c48b-49ba-47bb-b4ce-69acc9f47f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:58.603022 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1330c48b-49ba-47bb-b4ce-69acc9f47f93 None admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:58.603444 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1330c48b-49ba-47bb-b4ce-69acc9f47f93 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:58.603854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1330c48b-49ba-47bb-b4ce-69acc9f47f93 None admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:58.604767 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1330c48b-49ba-47bb-b4ce-69acc9f47f93 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:58.606331 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1330c48b-49ba-47bb-b4ce-69acc9f47f93 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:58.659497 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000ea4f9-8b0c-4b12-974a-0706e67a2f15 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:58.662355 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:58.688623 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 229/477] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Aug 29 22:28:58 2022] POST /identity/v3/users => generated 249 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:28:58.707410 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a33df-773c-4ac7-98db-8b8a7d2501e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:58.708028 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a33df-773c-4ac7-98db-8b8a7d2501e0 None admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:58.708374 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a33df-773c-4ac7-98db-8b8a7d2501e0 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:58.708654 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a33df-773c-4ac7-98db-8b8a7d2501e0 None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:58.709492 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a33df-773c-4ac7-98db-8b8a7d2501e0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:58.710920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a33df-773c-4ac7-98db-8b8a7d2501e0 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:58.750747 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.sql.core [None req-8b0a33df-773c-4ac7-98db-8b8a7d2501e0 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 29 22:28:58.750747 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 29 22:28:58.750747 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [parameters: {'id': '735b29d671944585ba0918d38e8131a9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 29 22:28:58.750747 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93271) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 29 22:28:58.773277 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-8b0a33df-773c-4ac7-98db-8b8a7d2501e0 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 29 22:28:58.773277 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:58.773277 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 22:28:58.773277 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 22:28:58.773277 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 22:28:58.773277 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 22:28:58.773277 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 22:28:58.773277 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 22:28:58.773277 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 22:28:58.773277 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application conn.query(q) Aug 29 22:28:58.773277 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 22:28:58.773277 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 22:28:58.773277 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 22:28:58.773277 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application result.read() Aug 29 22:28:58.773277 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 22:28:58.774222 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 22:28:58.774222 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 22:28:58.774222 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 22:28:58.774222 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 22:28:58.774222 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 22:28:58.774222 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 22:28:58.774222 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 22:28:58.774222 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 29 22:28:58.774222 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:28:58.774222 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 29 22:28:58.774222 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:28:58.774222 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:58.774222 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 29 22:28:58.774222 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 29 22:28:58.774222 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 29 22:28:58.774222 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return ref.to_dict() Aug 29 22:28:58.774222 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 22:28:58.774222 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application next(self.gen) Aug 29 22:28:58.775103 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 29 22:28:58.775103 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application yield resource Aug 29 22:28:58.775103 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 22:28:58.775103 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application next(self.gen) Aug 29 22:28:58.775103 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 29 22:28:58.775103 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application self.session.rollback() Aug 29 22:28:58.775103 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 29 22:28:58.775103 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application self.force_reraise() Aug 29 22:28:58.775103 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 29 22:28:58.775103 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise self.value Aug 29 22:28:58.775103 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 29 22:28:58.775103 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 29 22:28:58.775103 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 29 22:28:58.775103 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application session.commit() Aug 29 22:28:58.775103 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 29 22:28:58.775103 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 29 22:28:58.775103 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 29 22:28:58.775933 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application self._prepare_impl() Aug 29 22:28:58.775933 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 29 22:28:58.775933 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application self.session.flush() Aug 29 22:28:58.775933 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 29 22:28:58.775933 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application self._flush(objects) Aug 29 22:28:58.775933 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 29 22:28:58.775933 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 29 22:28:58.775933 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 29 22:28:58.775933 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application compat.raise_( Aug 29 22:28:58.775933 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 22:28:58.775933 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception Aug 29 22:28:58.775933 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 29 22:28:58.775933 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application flush_context.execute() Aug 29 22:28:58.775933 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 29 22:28:58.775933 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rec.execute(self) Aug 29 22:28:58.775933 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 29 22:28:58.775933 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 29 22:28:58.776850 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 29 22:28:58.776850 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 29 22:28:58.776850 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 29 22:28:58.776850 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 29 22:28:58.776850 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 29 22:28:58.776850 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 29 22:28:58.776850 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 29 22:28:58.776850 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 29 22:28:58.776850 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 29 22:28:58.776850 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 29 22:28:58.776850 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 29 22:28:58.776850 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 29 22:28:58.776850 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 29 22:28:58.776850 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 29 22:28:58.776850 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 22:28:58.776850 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception Aug 29 22:28:58.777656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 22:28:58.777656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 22:28:58.777656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 22:28:58.777656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 22:28:58.777656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 22:28:58.777656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 22:28:58.777656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 22:28:58.777656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application conn.query(q) Aug 29 22:28:58.777656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 22:28:58.777656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 22:28:58.777656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 22:28:58.777656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application result.read() Aug 29 22:28:58.777656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 22:28:58.777656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 22:28:58.777656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 22:28:58.777656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 22:28:58.778419 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 22:28:58.778419 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 22:28:58.778419 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 22:28:58.778419 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 22:28:58.778419 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 29 22:28:58.778419 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 29 22:28:58.778419 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [parameters: {'id': '735b29d671944585ba0918d38e8131a9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 29 22:28:58.778419 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 29 22:28:58.778419 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:28:58.778419 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:28:58.778419 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:28:58.778419 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:58.778419 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:58.778419 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:58.778419 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:58.778419 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:58.779191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:58.779191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:58.779191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:58.779191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:58.779191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:58.779191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:58.779191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 29 22:28:58.779191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 29 22:28:58.779191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:58.779191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:58.779191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 29 22:28:58.779191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 29 22:28:58.779191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 29 22:28:58.779191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 29 22:28:58.779191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 29 22:28:58.779191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:28:58.780138 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 249/478] 10.222.0.52 () {60 vars in 1131 bytes} [Mon Aug 29 22:28:58 2022] POST /identity/v3/roles => generated 144 bytes in 302 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:28:58.784860 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1330c48b-49ba-47bb-b4ce-69acc9f47f93 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:58.850097 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:58.947842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1330c48b-49ba-47bb-b4ce-69acc9f47f93 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:58.990525 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c638914f-34cd-4462-ad3f-263765930e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:58.991166 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c638914f-34cd-4462-ad3f-263765930e2c None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:58.991446 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c638914f-34cd-4462-ad3f-263765930e2c None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:58.991790 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c638914f-34cd-4462-ad3f-263765930e2c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:59.003966 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c638914f-34cd-4462-ad3f-263765930e2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:59.009220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-c638914f-34cd-4462-ad3f-263765930e2c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:28:59.009220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:59.009220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:59.009220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:59.009220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:59.009220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:59.009220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:59.009220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:59.009220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:59.009220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:59.009220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:59.009220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:59.009220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:59.009220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:59.009220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:59.009220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:59.010273 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:59.010273 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:59.010273 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:59.010273 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:59.010273 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:59.010273 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:59.010273 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:59.010273 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:59.010273 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:59.010273 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:59.010273 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:59.010273 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:59.010273 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:59.010273 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:59.010273 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:59.010273 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:59.011250 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:59.011250 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:59.011250 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:59.011250 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:59.011250 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:59.011250 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:59.011250 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:59.011250 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:59.011250 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:59.011250 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:59.011250 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:28:59.011250 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:28:59.016753 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 230/479] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Aug 29 22:28:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:59.016894 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a33df-773c-4ac7-98db-8b8a7d2501e0 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:59.129620 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a33df-773c-4ac7-98db-8b8a7d2501e0 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:59.167067 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d1e94-6a4f-47a7-8e0d-66fb3b08679b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:59.167680 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d1e94-6a4f-47a7-8e0d-66fb3b08679b None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:59.167958 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d1e94-6a4f-47a7-8e0d-66fb3b08679b None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:59.168226 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d1e94-6a4f-47a7-8e0d-66fb3b08679b None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:59.168914 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0d1e94-6a4f-47a7-8e0d-66fb3b08679b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:59.169215 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0d1e94-6a4f-47a7-8e0d-66fb3b08679b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:59.170582 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0d1e94-6a4f-47a7-8e0d-66fb3b08679b None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:59.180960 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 250/480] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:28:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:59.995290 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206faa6e-5bec-4127-a345-41bcb54a2174 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:59.995999 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206faa6e-5bec-4127-a345-41bcb54a2174 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:59.995999 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206faa6e-5bec-4127-a345-41bcb54a2174 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:59.997452 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 231/481] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:28:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:29:00.004092 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67bb009-9925-469d-ae39-7f849957b2fa None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:00.004672 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67bb009-9925-469d-ae39-7f849957b2fa None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:00.004672 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67bb009-9925-469d-ae39-7f849957b2fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:00.033771 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-d67bb009-9925-469d-ae39-7f849957b2fa None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:00.034979 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-d67bb009-9925-469d-ae39-7f849957b2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:00.069120 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:00.139337 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 251/482] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:29:00.292817 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05967f8f-0b46-4760-a222-16c51ff121a8 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:00.293117 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05967f8f-0b46-4760-a222-16c51ff121a8 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:00.293304 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05967f8f-0b46-4760-a222-16c51ff121a8 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:00.294902 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 232/483] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:29:00.300941 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4008bc-1509-4f4e-851b-15685722a961 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:00.301231 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4008bc-1509-4f4e-851b-15685722a961 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:00.301505 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4008bc-1509-4f4e-851b-15685722a961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:00.337849 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-ad4008bc-1509-4f4e-851b-15685722a961 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:00.339066 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-ad4008bc-1509-4f4e-851b-15685722a961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:00.418118 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 252/484] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:29:00.481181 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05967f8f-0b46-4760-a222-16c51ff121a8 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:00.527654 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d26fdd-6347-4550-bae1-984ef2ab4d7e None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:00.528000 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d26fdd-6347-4550-bae1-984ef2ab4d7e None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:00.528240 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d26fdd-6347-4550-bae1-984ef2ab4d7e None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:00.529978 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 253/485] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:29:00.535798 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb4384-967f-41cc-9003-75cdcfec5a2f None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:00.536035 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb4384-967f-41cc-9003-75cdcfec5a2f None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:00.536301 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb4384-967f-41cc-9003-75cdcfec5a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:00.555772 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-06fb4384-967f-41cc-9003-75cdcfec5a2f None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:00.557034 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-06fb4384-967f-41cc-9003-75cdcfec5a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:00.635629 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:00.657471 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05967f8f-0b46-4760-a222-16c51ff121a8 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:00.662499 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 254/486] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 22:29:00.712980 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e436f834-d483-48a0-b382-ca697e53a6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:00.713658 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e436f834-d483-48a0-b382-ca697e53a6ce None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:00.713944 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e436f834-d483-48a0-b382-ca697e53a6ce None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:00.714219 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e436f834-d483-48a0-b382-ca697e53a6ce None admin] PATH_INFO: `/v3/roles/service` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:00.733580 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e436f834-d483-48a0-b382-ca697e53a6ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:29:00.738564 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-e436f834-d483-48a0-b382-ca697e53a6ce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:29:00.738564 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:00.738564 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:00.738564 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:00.738564 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:00.738564 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:00.738564 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:00.738564 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:00.738564 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:00.738564 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:00.738564 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:00.738564 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:00.738564 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:29:00.738564 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:29:00.738564 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:29:00.738564 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:29:00.739423 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:29:00.739423 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:29:00.739423 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:29:00.739423 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:29:00.739423 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:00.739423 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:00.739423 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:00.739423 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:00.739423 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:00.739423 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:00.739423 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:00.739423 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:00.739423 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:00.739423 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:00.739423 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:00.739423 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:00.740631 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:00.740631 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:00.740631 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:00.740631 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:00.740631 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:29:00.740631 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:29:00.740631 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:29:00.740631 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:29:00.740631 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:29:00.740631 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:29:00.740631 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:29:00.740631 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:00.740631 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 233/487] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Aug 29 22:29:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:00.745687 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb4384-967f-41cc-9003-75cdcfec5a2f None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:00.774822 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:00.794433 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e436f834-d483-48a0-b382-ca697e53a6ce None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:00.870235 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb4384-967f-41cc-9003-75cdcfec5a2f None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:00.911044 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6da509-e7eb-4629-b6c4-d36eb4498121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:00.912158 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6da509-e7eb-4629-b6c4-d36eb4498121 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:00.912668 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6da509-e7eb-4629-b6c4-d36eb4498121 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:00.913158 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6da509-e7eb-4629-b6c4-d36eb4498121 None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:00.914016 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6da509-e7eb-4629-b6c4-d36eb4498121 None admin] RBAC: Adding query filter params (name=service) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:00.914512 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6da509-e7eb-4629-b6c4-d36eb4498121 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:00.916086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6da509-e7eb-4629-b6c4-d36eb4498121 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:00.927479 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 255/488] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Aug 29 22:29:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:00.934415 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6da509-e7eb-4629-b6c4-d36eb4498121 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:01.026889 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e436f834-d483-48a0-b382-ca697e53a6ce None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:01.074287 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30de84-8332-42a4-8465-4bb8bf18f79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:01.075689 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30de84-8332-42a4-8465-4bb8bf18f79a None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:01.076208 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30de84-8332-42a4-8465-4bb8bf18f79a None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:01.077119 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30de84-8332-42a4-8465-4bb8bf18f79a None admin] PATH_INFO: `/v3/roles/service` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:01.081136 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:01.093677 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f30de84-8332-42a4-8465-4bb8bf18f79a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:29:01.101293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-5f30de84-8332-42a4-8465-4bb8bf18f79a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:29:01.101293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:01.101293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:01.101293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:01.101293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:01.101293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:01.101293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:01.101293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:01.101293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:01.101293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:01.101293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:01.101293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:01.101293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:29:01.101293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:29:01.101293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:29:01.101293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:29:01.102222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:29:01.102222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:29:01.102222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:29:01.102222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:29:01.102222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:01.102222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:01.102222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:01.102222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:01.102222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:01.102222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:01.102222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:01.102222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:01.102222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:01.102222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:01.102222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:01.102222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:01.103378 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:01.103378 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:01.103378 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:01.103378 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:01.103378 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:29:01.103378 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:29:01.103378 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:29:01.103378 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:29:01.103378 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:29:01.103378 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:29:01.103378 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:29:01.103378 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:01.106048 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 234/489] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Aug 29 22:29:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:29:01.106421 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6da509-e7eb-4629-b6c4-d36eb4498121 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:01.107333 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30de84-8332-42a4-8465-4bb8bf18f79a None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:01.123966 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:01.153177 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a391a7-a1d3-4aa0-94f2-71a781981cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:01.153987 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a391a7-a1d3-4aa0-94f2-71a781981cd8 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:01.154284 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a391a7-a1d3-4aa0-94f2-71a781981cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:01.154562 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a391a7-a1d3-4aa0-94f2-71a781981cd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:01.170715 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a391a7-a1d3-4aa0-94f2-71a781981cd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:01.173109 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a391a7-a1d3-4aa0-94f2-71a781981cd8 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:01.193961 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-80a391a7-a1d3-4aa0-94f2-71a781981cd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:01.193961 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:01.193961 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:01.193961 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:01.193961 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:01.193961 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:01.193961 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:01.193961 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:01.193961 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:01.193961 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:01.193961 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:01.193961 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:01.193961 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:01.193961 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:01.193961 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:01.193961 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:01.194887 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:01.194887 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:01.194887 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:01.194887 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:01.194887 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:01.194887 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:01.194887 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:01.194887 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:01.194887 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:01.194887 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:01.194887 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:01.194887 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:01.194887 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:01.194887 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:01.194887 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:01.194887 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:01.194887 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:01.195728 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:01.195728 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:01.195728 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:01.200110 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 256/490] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:01.200984 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a391a7-a1d3-4aa0-94f2-71a781981cd8 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:01.242546 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30de84-8332-42a4-8465-4bb8bf18f79a None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:01.293282 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f905c01-8973-471f-9481-123561d1da41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:01.293915 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f905c01-8973-471f-9481-123561d1da41 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:01.294195 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f905c01-8973-471f-9481-123561d1da41 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:01.294470 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f905c01-8973-471f-9481-123561d1da41 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:01.310361 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f905c01-8973-471f-9481-123561d1da41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:29:01.318081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-8f905c01-8973-471f-9481-123561d1da41 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:29:01.318081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:01.318081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:01.318081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:01.318081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:01.318081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:01.318081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:01.318081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:01.318081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:01.318081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:01.318081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:01.318081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:01.318081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:29:01.318081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:29:01.318081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:29:01.318081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:29:01.318989 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:29:01.318989 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:29:01.318989 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:29:01.318989 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:29:01.318989 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:01.318989 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:01.318989 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:01.318989 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:01.318989 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:01.318989 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:01.318989 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:01.318989 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:01.318989 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:01.318989 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:01.318989 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:01.318989 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:01.319806 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:01.319806 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:01.319806 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:01.319806 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:01.319806 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:29:01.319806 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:29:01.319806 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:29:01.319806 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:29:01.319806 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:29:01.319806 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:29:01.319806 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:29:01.319806 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:01.324281 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 235/491] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Aug 29 22:29:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:01.325848 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f905c01-8973-471f-9481-123561d1da41 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:01.336294 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a391a7-a1d3-4aa0-94f2-71a781981cd8 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:01.385993 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8360a7c7-643b-457b-ab5a-04c420f34d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:01.386581 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8360a7c7-643b-457b-ab5a-04c420f34d69 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:01.386869 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8360a7c7-643b-457b-ab5a-04c420f34d69 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:01.387143 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8360a7c7-643b-457b-ab5a-04c420f34d69 None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:01.387814 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8360a7c7-643b-457b-ab5a-04c420f34d69 None admin] RBAC: Adding query filter params (name=service) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:01.388119 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8360a7c7-643b-457b-ab5a-04c420f34d69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:01.389529 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8360a7c7-643b-457b-ab5a-04c420f34d69 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:01.391950 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:01.406453 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:01.409272 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 257/492] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Aug 29 22:29:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:01.410319 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8360a7c7-643b-457b-ab5a-04c420f34d69 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:01.445019 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f905c01-8973-471f-9481-123561d1da41 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:01.483095 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40e59a6-1d2b-49f7-9e4f-b2be45d8f864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:01.484062 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e59a6-1d2b-49f7-9e4f-b2be45d8f864 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:01.484538 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e59a6-1d2b-49f7-9e4f-b2be45d8f864 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:01.485126 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e59a6-1d2b-49f7-9e4f-b2be45d8f864 None admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:01.485962 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40e59a6-1d2b-49f7-9e4f-b2be45d8f864 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:01.487026 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40e59a6-1d2b-49f7-9e4f-b2be45d8f864 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:01.488796 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40e59a6-1d2b-49f7-9e4f-b2be45d8f864 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:01.501624 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 236/493] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:01.503610 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40e59a6-1d2b-49f7-9e4f-b2be45d8f864 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:01.547297 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8360a7c7-643b-457b-ab5a-04c420f34d69 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:01.588433 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a7031-5c65-4bd1-ae4b-ff32fc8e8b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:01.589070 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a7031-5c65-4bd1-ae4b-ff32fc8e8b8a None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:01.589356 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a7031-5c65-4bd1-ae4b-ff32fc8e8b8a None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:01.589632 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a7031-5c65-4bd1-ae4b-ff32fc8e8b8a None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:01.590290 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69a7031-5c65-4bd1-ae4b-ff32fc8e8b8a None admin] RBAC: Adding query filter params (name=service) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:01.590592 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69a7031-5c65-4bd1-ae4b-ff32fc8e8b8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:01.592009 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69a7031-5c65-4bd1-ae4b-ff32fc8e8b8a None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:01.600665 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 258/494] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Aug 29 22:29:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:01.601517 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a7031-5c65-4bd1-ae4b-ff32fc8e8b8a None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:01.617549 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40e59a6-1d2b-49f7-9e4f-b2be45d8f864 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:01.646359 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:01.655161 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a53bc0-0afc-4573-9bca-6c5695fc5faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:01.655848 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a53bc0-0afc-4573-9bca-6c5695fc5faf None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:01.656126 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a53bc0-0afc-4573-9bca-6c5695fc5faf None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:01.656541 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a53bc0-0afc-4573-9bca-6c5695fc5faf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:01.662451 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:01.668236 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a53bc0-0afc-4573-9bca-6c5695fc5faf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:01.669862 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a53bc0-0afc-4573-9bca-6c5695fc5faf None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:01.686113 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-34a53bc0-0afc-4573-9bca-6c5695fc5faf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:01.686113 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:01.686113 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:01.686113 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:01.686113 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:01.686113 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:01.686113 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:01.686113 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:01.686113 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:01.686113 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:01.686113 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:01.686113 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:01.686113 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:01.686113 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:01.686113 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:01.686113 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:01.687583 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:01.687583 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:01.687583 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:01.687583 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:01.687583 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:01.687583 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:01.687583 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:01.687583 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:01.687583 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:01.687583 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:01.687583 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:01.687583 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:01.687583 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:01.687583 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:01.687583 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:01.687583 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:01.687583 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:01.689185 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:01.689185 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:01.689185 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:01.692128 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 237/495] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:01.693412 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd769a-38e0-4d87-a33a-696a440bebf4 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:01.693948 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd769a-38e0-4d87-a33a-696a440bebf4 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:01.693948 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd769a-38e0-4d87-a33a-696a440bebf4 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:01.695343 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 238/496] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:29:01.697846 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd769a-38e0-4d87-a33a-696a440bebf4 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:01.714473 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a7031-5c65-4bd1-ae4b-ff32fc8e8b8a None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:01.750895 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717a38cb-81cb-4bf6-bf3d-2bcbd979e35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:01.751524 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a38cb-81cb-4bf6-bf3d-2bcbd979e35e None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:01.751805 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a38cb-81cb-4bf6-bf3d-2bcbd979e35e None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:01.752254 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a38cb-81cb-4bf6-bf3d-2bcbd979e35e None admin] PATH_INFO: `/v3/users/glance` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:01.760263 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717a38cb-81cb-4bf6-bf3d-2bcbd979e35e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:01.761818 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717a38cb-81cb-4bf6-bf3d-2bcbd979e35e None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:01.775492 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-717a38cb-81cb-4bf6-bf3d-2bcbd979e35e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 29 22:29:01.775492 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:01.775492 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:01.775492 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:01.775492 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:01.775492 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:01.775492 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:01.775492 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:01.775492 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:01.775492 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:01.775492 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:01.775492 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:01.775492 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:29:01.775492 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:29:01.775492 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:29:01.775492 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:29:01.777497 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:01.777497 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:01.777497 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:29:01.777497 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:01.777497 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:29:01.777497 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:01.777497 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:01.777497 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:01.777497 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:01.777497 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:01.777497 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:01.777497 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:01.777497 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:01.777497 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:01.777497 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:01.777497 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:01.777497 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:01.777497 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:01.778763 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:01.778763 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:01.778763 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:29:01.778763 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:29:01.778763 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:29:01.778763 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:29:01.778763 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:29:01.778763 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:29:01.778763 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 29 22:29:01.778763 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:01.778763 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 259/497] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:29:01 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:01.778763 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717a38cb-81cb-4bf6-bf3d-2bcbd979e35e None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:01.822083 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd769a-38e0-4d87-a33a-696a440bebf4 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:01.858027 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a5c49b-9a32-4b57-8cca-f61313eb916b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:01.858681 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5c49b-9a32-4b57-8cca-f61313eb916b None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:01.858952 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5c49b-9a32-4b57-8cca-f61313eb916b None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:01.859229 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5c49b-9a32-4b57-8cca-f61313eb916b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:01.870199 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a5c49b-9a32-4b57-8cca-f61313eb916b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:01.871658 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a5c49b-9a32-4b57-8cca-f61313eb916b None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:01.883592 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717a38cb-81cb-4bf6-bf3d-2bcbd979e35e None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:01.886521 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-f8a5c49b-9a32-4b57-8cca-f61313eb916b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:01.886521 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:01.886521 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:01.886521 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:01.886521 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:01.886521 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:01.886521 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:01.886521 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:01.886521 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:01.886521 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:01.886521 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:01.886521 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:01.886521 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:01.886521 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:01.886521 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:01.886521 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:01.887395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:01.887395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:01.887395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:01.887395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:01.887395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:01.887395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:01.887395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:01.887395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:01.887395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:01.887395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:01.887395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:01.887395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:01.887395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:01.887395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:01.887395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:01.887395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:01.887395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:01.888239 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:01.888239 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:01.888239 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:01.888239 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 239/498] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:01.888865 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3e7e2-2ee4-41c4-b4a4-9511fd3968af None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:01.889100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3e7e2-2ee4-41c4-b4a4-9511fd3968af None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:01.889429 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3e7e2-2ee4-41c4-b4a4-9511fd3968af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:01.903767 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:01.908632 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-0bb3e7e2-2ee4-41c4-b4a4-9511fd3968af None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:01.909848 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-0bb3e7e2-2ee4-41c4-b4a4-9511fd3968af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:01.920317 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39aba30-ae4f-4d22-b091-4f2a8bd0225b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:01.920910 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39aba30-ae4f-4d22-b091-4f2a8bd0225b None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:01.921118 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39aba30-ae4f-4d22-b091-4f2a8bd0225b None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:01.921406 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39aba30-ae4f-4d22-b091-4f2a8bd0225b None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:01.922014 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39aba30-ae4f-4d22-b091-4f2a8bd0225b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:01.922312 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39aba30-ae4f-4d22-b091-4f2a8bd0225b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:01.923980 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39aba30-ae4f-4d22-b091-4f2a8bd0225b None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:01.936502 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 260/499] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:01.937590 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39aba30-ae4f-4d22-b091-4f2a8bd0225b None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:01.938032 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:01.987227 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 240/500] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:29:01.988557 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80edbf18-cd8b-4557-8e8c-7edf3e950457 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:01.988907 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80edbf18-cd8b-4557-8e8c-7edf3e950457 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:01.989240 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80edbf18-cd8b-4557-8e8c-7edf3e950457 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:01.990714 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 241/501] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:29:01.991298 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80edbf18-cd8b-4557-8e8c-7edf3e950457 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:02.046782 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39aba30-ae4f-4d22-b091-4f2a8bd0225b None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:02.085374 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173e13d9-a9d6-4ca3-a0e4-83c086878b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:02.086022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173e13d9-a9d6-4ca3-a0e4-83c086878b56 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:02.086555 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173e13d9-a9d6-4ca3-a0e4-83c086878b56 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:02.086555 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173e13d9-a9d6-4ca3-a0e4-83c086878b56 None admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:02.087225 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173e13d9-a9d6-4ca3-a0e4-83c086878b56 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:02.087533 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173e13d9-a9d6-4ca3-a0e4-83c086878b56 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:02.089015 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173e13d9-a9d6-4ca3-a0e4-83c086878b56 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:02.114865 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80edbf18-cd8b-4557-8e8c-7edf3e950457 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:02.132123 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 261/502] 10.222.0.52 () {56 vars in 1137 bytes} [Mon Aug 29 22:29:01 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:02.132123 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173e13d9-a9d6-4ca3-a0e4-83c086878b56 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:02.156614 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81383dc-3ab0-4806-ae42-db9226352247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:02.157176 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81383dc-3ab0-4806-ae42-db9226352247 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:02.157456 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81383dc-3ab0-4806-ae42-db9226352247 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:02.157723 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81383dc-3ab0-4806-ae42-db9226352247 None admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:02.158387 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81383dc-3ab0-4806-ae42-db9226352247 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:02.158688 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81383dc-3ab0-4806-ae42-db9226352247 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:02.160044 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81383dc-3ab0-4806-ae42-db9226352247 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:02.168090 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:02.172528 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 242/503] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:02.178888 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f55547-24e2-4e33-ab02-41468996371c None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:02.179178 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f55547-24e2-4e33-ab02-41468996371c None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:02.183302 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f55547-24e2-4e33-ab02-41468996371c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:02.198495 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-05f55547-24e2-4e33-ab02-41468996371c None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:02.199676 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-05f55547-24e2-4e33-ab02-41468996371c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:02.216885 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:02.243013 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173e13d9-a9d6-4ca3-a0e4-83c086878b56 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:02.274147 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 243/504] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:29:02.275017 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f55547-24e2-4e33-ab02-41468996371c None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:02.282788 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d3fbbd-5525-40be-ab6f-5f7e73bdbb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:02.284091 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d3fbbd-5525-40be-ab6f-5f7e73bdbb00 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:02.284826 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d3fbbd-5525-40be-ab6f-5f7e73bdbb00 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:02.284826 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d3fbbd-5525-40be-ab6f-5f7e73bdbb00 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:02.292032 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d3fbbd-5525-40be-ab6f-5f7e73bdbb00 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:02.293522 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d3fbbd-5525-40be-ab6f-5f7e73bdbb00 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:02.305935 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-88d3fbbd-5525-40be-ab6f-5f7e73bdbb00 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 22:29:02.305935 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:02.305935 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:02.305935 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:02.305935 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:02.305935 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:02.305935 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:02.305935 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:02.305935 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:02.305935 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:02.305935 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:02.305935 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:02.305935 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:29:02.305935 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:29:02.305935 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:29:02.305935 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:29:02.306884 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:02.306884 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:02.306884 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:29:02.306884 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:02.306884 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:29:02.306884 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:02.306884 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:02.306884 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:02.306884 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:02.306884 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:02.306884 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:02.306884 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:02.306884 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:02.306884 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:02.306884 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:02.306884 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:02.306884 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:02.306884 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:02.307776 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:02.307776 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:02.307776 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:29:02.307776 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:29:02.307776 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:29:02.307776 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:29:02.307776 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:29:02.307776 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:29:02.307776 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 22:29:02.307776 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:02.307776 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 262/505] 10.222.0.52 () {56 vars in 1093 bytes} [Mon Aug 29 22:29:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:02.307776 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d3fbbd-5525-40be-ab6f-5f7e73bdbb00 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:02.390609 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f55547-24e2-4e33-ab02-41468996371c None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:02.427520 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b67c4-673a-41dc-a96f-f043f7260766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:02.428058 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b67c4-673a-41dc-a96f-f043f7260766 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:02.428320 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b67c4-673a-41dc-a96f-f043f7260766 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:02.428589 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b67c4-673a-41dc-a96f-f043f7260766 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:02.432950 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:02.438120 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d3fbbd-5525-40be-ab6f-5f7e73bdbb00 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:02.451689 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:02.475088 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc4033b-3338-4ebe-b2cd-a9e382e7de5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:02.475711 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4033b-3338-4ebe-b2cd-a9e382e7de5e None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:02.475987 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4033b-3338-4ebe-b2cd-a9e382e7de5e None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:02.476258 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4033b-3338-4ebe-b2cd-a9e382e7de5e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:02.484075 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc4033b-3338-4ebe-b2cd-a9e382e7de5e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:02.485873 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc4033b-3338-4ebe-b2cd-a9e382e7de5e None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:02.501358 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-4dc4033b-3338-4ebe-b2cd-a9e382e7de5e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 22:29:02.501358 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:02.501358 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:02.501358 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:02.501358 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:02.501358 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:02.501358 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:02.501358 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:02.501358 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:02.501358 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:02.501358 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:02.501358 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:02.501358 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:29:02.501358 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:29:02.501358 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:29:02.501358 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:29:02.502905 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:02.502905 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:02.502905 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:29:02.502905 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:02.502905 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:29:02.502905 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:02.502905 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:02.502905 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:02.502905 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:02.502905 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:02.502905 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:02.502905 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:02.502905 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:02.502905 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:02.502905 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:02.502905 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:02.502905 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:02.502905 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:02.508868 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:02.508868 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:02.508868 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:29:02.508868 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:29:02.508868 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:29:02.508868 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:29:02.508868 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:29:02.508868 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:29:02.508868 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 22:29:02.508868 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:02.509825 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0b67c4-673a-41dc-a96f-f043f7260766 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:02.509899 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 263/506] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:29:02 2022] GET /identity/v3/users/cinder => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:02.509899 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc4033b-3338-4ebe-b2cd-a9e382e7de5e None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:02.510026 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0b67c4-673a-41dc-a96f-f043f7260766 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:02.613208 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 244/507] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:29:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:29:02.614141 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b67c4-673a-41dc-a96f-f043f7260766 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:02.630310 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc4033b-3338-4ebe-b2cd-a9e382e7de5e None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:02.665367 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153974f0-ca33-4d3d-aa15-aba996febb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:02.665987 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153974f0-ca33-4d3d-aa15-aba996febb94 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:02.666262 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153974f0-ca33-4d3d-aa15-aba996febb94 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:02.666530 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153974f0-ca33-4d3d-aa15-aba996febb94 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:02.677762 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:02.678430 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153974f0-ca33-4d3d-aa15-aba996febb94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:29:02.683833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-153974f0-ca33-4d3d-aa15-aba996febb94 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:29:02.683833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:02.683833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:02.683833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:02.683833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:02.683833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:02.683833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:02.683833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:02.683833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:02.683833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:02.683833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:02.683833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:02.683833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:29:02.683833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:29:02.683833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:29:02.683833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:29:02.691335 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:29:02.691335 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:29:02.691335 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:29:02.691335 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:29:02.691335 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:02.691335 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:02.691335 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:02.691335 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:02.691335 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:02.691335 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:02.691335 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:02.691335 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:02.691335 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:02.691335 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:02.691335 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:02.691335 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:02.692345 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:02.692345 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:02.692345 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:02.692345 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:02.692345 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:29:02.692345 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:29:02.692345 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:29:02.692345 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:29:02.692345 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:29:02.692345 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:29:02.692345 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:29:02.692345 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:02.692345 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 264/508] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Aug 29 22:29:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:02.692345 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153974f0-ca33-4d3d-aa15-aba996febb94 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:02.708573 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:02.729793 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b67c4-673a-41dc-a96f-f043f7260766 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:02.765509 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e96677-39e9-4d90-825c-d1a9a5f72061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:02.766100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e96677-39e9-4d90-825c-d1a9a5f72061 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:02.766383 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e96677-39e9-4d90-825c-d1a9a5f72061 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:02.767057 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e96677-39e9-4d90-825c-d1a9a5f72061 None admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:02.768547 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e96677-39e9-4d90-825c-d1a9a5f72061 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:02.768885 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e96677-39e9-4d90-825c-d1a9a5f72061 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:02.770323 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e96677-39e9-4d90-825c-d1a9a5f72061 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:02.793069 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153974f0-ca33-4d3d-aa15-aba996febb94 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:02.805447 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 245/509] 10.222.0.52 () {56 vars in 1133 bytes} [Mon Aug 29 22:29:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:02.806268 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e96677-39e9-4d90-825c-d1a9a5f72061 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:02.833569 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3823e708-17a4-4ff6-9d00-85ad10808f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:02.836126 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3823e708-17a4-4ff6-9d00-85ad10808f4c None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:02.836415 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3823e708-17a4-4ff6-9d00-85ad10808f4c None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:02.836692 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3823e708-17a4-4ff6-9d00-85ad10808f4c None admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:02.838744 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3823e708-17a4-4ff6-9d00-85ad10808f4c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:02.839271 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3823e708-17a4-4ff6-9d00-85ad10808f4c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:02.841934 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3823e708-17a4-4ff6-9d00-85ad10808f4c None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:02.867163 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 265/510] 10.222.0.52 () {56 vars in 1137 bytes} [Mon Aug 29 22:29:02 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:02.867985 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3823e708-17a4-4ff6-9d00-85ad10808f4c None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:02.912450 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e96677-39e9-4d90-825c-d1a9a5f72061 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:02.922931 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:02.946397 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:02.947658 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a8e3b-7055-4583-b63b-cc24f69eb8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:02.948275 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a8e3b-7055-4583-b63b-cc24f69eb8b4 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:02.948651 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a8e3b-7055-4583-b63b-cc24f69eb8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:02.948951 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a8e3b-7055-4583-b63b-cc24f69eb8b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:02.959800 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a8e3b-7055-4583-b63b-cc24f69eb8b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:02.961424 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a8e3b-7055-4583-b63b-cc24f69eb8b4 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:02.973097 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3823e708-17a4-4ff6-9d00-85ad10808f4c None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:02.976389 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-9e6a8e3b-7055-4583-b63b-cc24f69eb8b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:02.976389 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:02.976389 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:02.976389 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:02.976389 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:02.976389 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:02.976389 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:02.976389 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:02.976389 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:02.976389 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:02.976389 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:02.976389 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:02.976389 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:02.976389 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:02.976389 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:02.976389 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:02.977561 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:02.977561 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:02.977561 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:02.977561 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:02.977561 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:02.977561 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:02.977561 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:02.977561 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:02.977561 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:02.977561 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:02.977561 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:02.977561 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:02.977561 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:02.977561 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:02.977561 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:02.977561 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:02.977561 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:02.978462 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:02.978462 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:02.978462 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:02.978462 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 246/511] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:02.979677 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a8e3b-7055-4583-b63b-cc24f69eb8b4 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:03.014194 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39388a-7358-4556-8508-7ae5e6d2bbbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:03.015003 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39388a-7358-4556-8508-7ae5e6d2bbbe None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:03.015288 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39388a-7358-4556-8508-7ae5e6d2bbbe None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:03.015561 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39388a-7358-4556-8508-7ae5e6d2bbbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:03.026465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f39388a-7358-4556-8508-7ae5e6d2bbbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:03.027941 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f39388a-7358-4556-8508-7ae5e6d2bbbe None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:03.043448 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-2f39388a-7358-4556-8508-7ae5e6d2bbbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:03.043448 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:03.043448 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:03.043448 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:03.043448 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:03.043448 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:03.043448 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:03.043448 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:03.043448 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:03.043448 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:03.043448 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:03.043448 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:03.043448 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:03.043448 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:03.043448 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:03.043448 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:03.044571 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:03.044571 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:03.044571 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:03.044571 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:03.044571 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:03.044571 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:03.044571 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:03.044571 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:03.044571 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:03.044571 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:03.044571 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:03.044571 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:03.044571 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:03.044571 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:03.044571 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:03.044571 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:03.044571 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:03.046122 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:03.046122 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:03.046122 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:03.046122 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 266/512] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:03.046122 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39388a-7358-4556-8508-7ae5e6d2bbbe None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:03.095851 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a8e3b-7055-4583-b63b-cc24f69eb8b4 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:03.131907 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de1113d-b450-40c7-892c-d9a67b5573b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:03.132571 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1113d-b450-40c7-892c-d9a67b5573b5 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:03.132835 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1113d-b450-40c7-892c-d9a67b5573b5 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:03.133114 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1113d-b450-40c7-892c-d9a67b5573b5 None admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:03.133775 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de1113d-b450-40c7-892c-d9a67b5573b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:03.134082 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de1113d-b450-40c7-892c-d9a67b5573b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:03.135475 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de1113d-b450-40c7-892c-d9a67b5573b5 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:03.143876 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 247/513] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Aug 29 22:29:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:03.144688 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de1113d-b450-40c7-892c-d9a67b5573b5 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:03.149754 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39388a-7358-4556-8508-7ae5e6d2bbbe None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:03.175551 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:03.176390 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:03.184490 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b5a20-b9e3-40f1-b473-d958c0cc8f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:03.185171 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b5a20-b9e3-40f1-b473-d958c0cc8f58 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:03.193053 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b5a20-b9e3-40f1-b473-d958c0cc8f58 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:03.193053 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b5a20-b9e3-40f1-b473-d958c0cc8f58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:03.250913 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de1113d-b450-40c7-892c-d9a67b5573b5 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:03.254053 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898b5a20-b9e3-40f1-b473-d958c0cc8f58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:03.255586 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898b5a20-b9e3-40f1-b473-d958c0cc8f58 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:03.285355 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6b3a0-d420-406f-b4fe-9aceba305031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:03.285958 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6b3a0-d420-406f-b4fe-9aceba305031 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:03.286236 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6b3a0-d420-406f-b4fe-9aceba305031 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:03.286507 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6b3a0-d420-406f-b4fe-9aceba305031 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:03.359473 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db6b3a0-d420-406f-b4fe-9aceba305031 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:03.362812 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db6b3a0-d420-406f-b4fe-9aceba305031 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:03.362950 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 267/514] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:29:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:29:03.364799 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b5a20-b9e3-40f1-b473-d958c0cc8f58 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:03.392538 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:03.394483 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:03.467276 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b5a20-b9e3-40f1-b473-d958c0cc8f58 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:03.467536 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 248/515] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:29:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:29:03.468671 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6b3a0-d420-406f-b4fe-9aceba305031 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:03.503298 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c909d31-c910-4c27-a4ab-e2169f730a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:03.503825 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c909d31-c910-4c27-a4ab-e2169f730a7c None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:03.504138 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c909d31-c910-4c27-a4ab-e2169f730a7c None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:03.504443 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c909d31-c910-4c27-a4ab-e2169f730a7c None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:03.505143 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c909d31-c910-4c27-a4ab-e2169f730a7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:03.505478 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c909d31-c910-4c27-a4ab-e2169f730a7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:03.506883 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c909d31-c910-4c27-a4ab-e2169f730a7c None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:03.518880 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 268/516] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:03.519695 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c909d31-c910-4c27-a4ab-e2169f730a7c None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:03.573756 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6b3a0-d420-406f-b4fe-9aceba305031 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:03.598904 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:03.607952 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:03.609536 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a54c67-3541-49a1-8ab3-084d00788c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:03.610146 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a54c67-3541-49a1-8ab3-084d00788c8b None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:03.610422 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a54c67-3541-49a1-8ab3-084d00788c8b None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:03.610692 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a54c67-3541-49a1-8ab3-084d00788c8b None admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:03.611526 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a54c67-3541-49a1-8ab3-084d00788c8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:03.612075 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a54c67-3541-49a1-8ab3-084d00788c8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:03.613121 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a54c67-3541-49a1-8ab3-084d00788c8b None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:03.622637 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c909d31-c910-4c27-a4ab-e2169f730a7c None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:03.624422 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 249/517] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:03.625265 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a54c67-3541-49a1-8ab3-084d00788c8b None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:03.657851 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d663f-361b-440b-864b-83e305905c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:03.658797 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d663f-361b-440b-864b-83e305905c4f None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:03.659085 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d663f-361b-440b-864b-83e305905c4f None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:03.659365 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d663f-361b-440b-864b-83e305905c4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:03.670695 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9d663f-361b-440b-864b-83e305905c4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:03.672223 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9d663f-361b-440b-864b-83e305905c4f None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:03.687155 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-4c9d663f-361b-440b-864b-83e305905c4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:03.687155 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:03.687155 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:03.687155 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:03.687155 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:03.687155 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:03.687155 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:03.687155 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:03.687155 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:03.687155 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:03.687155 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:03.687155 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:03.687155 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:03.687155 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:03.687155 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:03.687155 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:03.688065 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:03.688065 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:03.688065 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:03.688065 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:03.688065 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:03.688065 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:03.688065 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:03.688065 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:03.688065 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:03.688065 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:03.688065 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:03.688065 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:03.688065 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:03.688065 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:03.688065 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:03.688065 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:03.688065 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:03.695005 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:03.695005 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:03.695005 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:03.695005 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 269/518] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:03.695005 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d663f-361b-440b-864b-83e305905c4f None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:03.726992 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a54c67-3541-49a1-8ab3-084d00788c8b None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:03.761603 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c3094-5232-4a9d-9b2f-f677084f513c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:03.762154 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c3094-5232-4a9d-9b2f-f677084f513c None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:03.762439 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c3094-5232-4a9d-9b2f-f677084f513c None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:03.762715 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c3094-5232-4a9d-9b2f-f677084f513c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:03.773389 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3c3094-5232-4a9d-9b2f-f677084f513c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:03.774839 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3c3094-5232-4a9d-9b2f-f677084f513c None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:03.789187 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-5b3c3094-5232-4a9d-9b2f-f677084f513c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:03.789187 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:03.789187 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:03.789187 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:03.789187 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:03.789187 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:03.789187 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:03.789187 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:03.789187 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:03.789187 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:03.789187 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:03.789187 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:03.789187 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:03.789187 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:03.789187 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:03.789187 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:03.790103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:03.790103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:03.790103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:03.790103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:03.790103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:03.790103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:03.790103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:03.790103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:03.790103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:03.790103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:03.790103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:03.790103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:03.790103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:03.790103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:03.790103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:03.790103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:03.790103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:03.790948 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:03.790948 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:03.790948 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:03.791124 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d663f-361b-440b-864b-83e305905c4f None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:03.791196 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 250/519] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:03.791196 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c3094-5232-4a9d-9b2f-f677084f513c None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:03.825368 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b583e6-23a3-4537-98c2-ea075e0ece0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:03.826002 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b583e6-23a3-4537-98c2-ea075e0ece0c None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:03.826286 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b583e6-23a3-4537-98c2-ea075e0ece0c None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:03.826563 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b583e6-23a3-4537-98c2-ea075e0ece0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:03.830569 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:03.833909 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:03.837701 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b583e6-23a3-4537-98c2-ea075e0ece0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:03.839148 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b583e6-23a3-4537-98c2-ea075e0ece0c None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:03.853428 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-36b583e6-23a3-4537-98c2-ea075e0ece0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:03.853428 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:03.853428 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:03.853428 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:03.853428 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:03.853428 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:03.853428 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:03.853428 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:03.853428 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:03.853428 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:03.853428 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:03.853428 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:03.853428 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:03.853428 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:03.853428 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:03.853428 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:03.854295 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:03.854295 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:03.854295 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:03.854295 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:03.854295 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:03.854295 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:03.854295 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:03.854295 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:03.854295 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:03.854295 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:03.854295 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:03.854295 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:03.854295 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:03.854295 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:03.854295 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:03.854295 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:03.854295 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:03.855124 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:03.855124 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:03.855124 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:03.855124 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 270/520] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:03.855345 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b583e6-23a3-4537-98c2-ea075e0ece0c None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:03.899831 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c3094-5232-4a9d-9b2f-f677084f513c None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:03.933369 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987b669-24d4-4c0f-a16f-16649e976af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:03.933963 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987b669-24d4-4c0f-a16f-16649e976af6 None admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:03.934226 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987b669-24d4-4c0f-a16f-16649e976af6 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:03.934492 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987b669-24d4-4c0f-a16f-16649e976af6 None admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:03.935120 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9987b669-24d4-4c0f-a16f-16649e976af6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:03.936551 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9987b669-24d4-4c0f-a16f-16649e976af6 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:03.958404 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b583e6-23a3-4537-98c2-ea075e0ece0c None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:03.991932 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88af113-f87b-4a39-87ae-b099536298e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:03.992547 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88af113-f87b-4a39-87ae-b099536298e8 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:03.992859 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88af113-f87b-4a39-87ae-b099536298e8 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:03.993131 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88af113-f87b-4a39-87ae-b099536298e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:03.999731 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88af113-f87b-4a39-87ae-b099536298e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:04.001260 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88af113-f87b-4a39-87ae-b099536298e8 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:04.017377 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-e88af113-f87b-4a39-87ae-b099536298e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:04.017377 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:04.017377 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:04.017377 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:04.017377 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:04.017377 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:04.017377 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:04.017377 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:04.017377 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:04.017377 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:04.017377 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:04.017377 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:04.017377 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:29:04.017377 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:29:04.017377 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:29:04.018481 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:29:04.018481 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:04.018481 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:04.018481 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:04.018481 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:04.018481 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:04.018481 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:04.018481 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:04.018481 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:04.018481 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:04.018481 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:04.018481 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:04.018481 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:04.018481 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:04.018481 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:04.018481 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:04.018481 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:04.019344 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:29:04.019344 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:29:04.019344 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:29:04.019344 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:29:04.019344 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:29:04.019344 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:29:04.019344 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:04.019344 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:04.019344 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 271/521] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:29:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:04.019344 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88af113-f87b-4a39-87ae-b099536298e8 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:04.041894 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 251/522] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Aug 29 22:29:03 2022] POST /identity/v3/users => generated 247 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:29:04.042803 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987b669-24d4-4c0f-a16f-16649e976af6 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:04.069752 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:04.127730 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88af113-f87b-4a39-87ae-b099536298e8 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:04.164209 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d8e23f-dca7-447b-8c3f-c19652fc1c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:04.165029 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d8e23f-dca7-447b-8c3f-c19652fc1c0d None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:04.165111 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d8e23f-dca7-447b-8c3f-c19652fc1c0d None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:04.165386 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d8e23f-dca7-447b-8c3f-c19652fc1c0d None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:04.166037 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d8e23f-dca7-447b-8c3f-c19652fc1c0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:04.166345 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d8e23f-dca7-447b-8c3f-c19652fc1c0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:04.167752 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d8e23f-dca7-447b-8c3f-c19652fc1c0d None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:04.168552 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:04.180248 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987b669-24d4-4c0f-a16f-16649e976af6 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:04.182030 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 272/523] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:04.182812 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d8e23f-dca7-447b-8c3f-c19652fc1c0d None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:04.224964 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd65310-39d3-43ff-88a6-49176c14ec6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:04.225577 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd65310-39d3-43ff-88a6-49176c14ec6e None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:04.225861 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd65310-39d3-43ff-88a6-49176c14ec6e None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:04.226195 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd65310-39d3-43ff-88a6-49176c14ec6e None admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:04.226938 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd65310-39d3-43ff-88a6-49176c14ec6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:04.227256 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd65310-39d3-43ff-88a6-49176c14ec6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:04.228631 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd65310-39d3-43ff-88a6-49176c14ec6e None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:04.244579 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 252/524] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:04.245784 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd65310-39d3-43ff-88a6-49176c14ec6e None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:04.301648 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d8e23f-dca7-447b-8c3f-c19652fc1c0d None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:04.315546 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:04.340524 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662fd073-893b-4f37-addf-656501680da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:04.341268 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662fd073-893b-4f37-addf-656501680da3 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:04.341551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662fd073-893b-4f37-addf-656501680da3 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:04.341824 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662fd073-893b-4f37-addf-656501680da3 None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:04.342471 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662fd073-893b-4f37-addf-656501680da3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:04.342776 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662fd073-893b-4f37-addf-656501680da3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:04.344280 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662fd073-893b-4f37-addf-656501680da3 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:04.354831 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd65310-39d3-43ff-88a6-49176c14ec6e None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:04.356279 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 273/525] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:04.360889 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662fd073-893b-4f37-addf-656501680da3 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:04.405003 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1ca744-d276-4120-a16a-d2b15ef5b152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:04.405615 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1ca744-d276-4120-a16a-d2b15ef5b152 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:04.405895 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1ca744-d276-4120-a16a-d2b15ef5b152 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:04.406169 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1ca744-d276-4120-a16a-d2b15ef5b152 None admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:04.406862 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1ca744-d276-4120-a16a-d2b15ef5b152 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:04.407174 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1ca744-d276-4120-a16a-d2b15ef5b152 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:04.409636 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1ca744-d276-4120-a16a-d2b15ef5b152 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:04.430925 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 253/526] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Aug 29 22:29:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:04.436500 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1ca744-d276-4120-a16a-d2b15ef5b152 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:04.454137 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:04.474030 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662fd073-893b-4f37-addf-656501680da3 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:04.511919 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceeaf49-480e-4518-ae59-6d2043541bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:04.512949 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceeaf49-480e-4518-ae59-6d2043541bb1 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:04.513234 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceeaf49-480e-4518-ae59-6d2043541bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:04.513508 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceeaf49-480e-4518-ae59-6d2043541bb1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:04.522250 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceeaf49-480e-4518-ae59-6d2043541bb1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:04.523685 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceeaf49-480e-4518-ae59-6d2043541bb1 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:04.536058 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-9ceeaf49-480e-4518-ae59-6d2043541bb1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 22:29:04.536058 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:04.536058 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:04.536058 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:04.536058 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:04.536058 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:04.536058 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:04.536058 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:04.536058 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:04.536058 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:04.536058 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:04.536058 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:04.536058 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:29:04.536058 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:29:04.536058 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:29:04.536058 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:29:04.537049 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:04.537049 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:04.537049 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:29:04.537049 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:04.537049 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:29:04.537049 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:04.537049 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:04.537049 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:04.537049 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:04.537049 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:04.537049 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:04.537049 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:04.537049 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:04.537049 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:04.537049 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:04.537049 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:04.537049 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:04.537049 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:04.537927 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:04.537927 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:04.537927 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:29:04.537927 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:29:04.537927 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:29:04.537927 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:29:04.537927 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:29:04.537927 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:29:04.537927 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 22:29:04.537927 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:04.545076 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 274/527] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Aug 29 22:29:04 2022] GET /identity/v3/users/neutron => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:04.545076 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceeaf49-480e-4518-ae59-6d2043541bb1 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:04.556744 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1ca744-d276-4120-a16a-d2b15ef5b152 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:04.586637 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:04.595140 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c97755-7c74-4abb-ba7d-26be66ae8a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:04.604232 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c97755-7c74-4abb-ba7d-26be66ae8a29 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:04.604232 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c97755-7c74-4abb-ba7d-26be66ae8a29 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:04.604232 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c97755-7c74-4abb-ba7d-26be66ae8a29 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:04.604232 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c97755-7c74-4abb-ba7d-26be66ae8a29 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:04.605781 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c97755-7c74-4abb-ba7d-26be66ae8a29 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:04.619209 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-25c97755-7c74-4abb-ba7d-26be66ae8a29 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:04.619209 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:04.619209 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:04.619209 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:04.619209 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:04.619209 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:04.619209 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:04.619209 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:04.619209 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:04.619209 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:04.619209 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:04.619209 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:04.619209 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:29:04.619209 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:29:04.619209 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:29:04.620277 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:29:04.620277 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:04.620277 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:04.620277 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:04.620277 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:04.620277 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:04.620277 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:04.620277 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:04.620277 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:04.620277 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:04.620277 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:04.620277 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:04.620277 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:04.620277 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:04.620277 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:04.620277 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:04.620277 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:04.625707 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:29:04.625707 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:29:04.625707 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:29:04.625707 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:29:04.625707 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:29:04.625707 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:29:04.625707 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:04.625707 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:04.625707 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 254/528] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:29:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:04.625707 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c97755-7c74-4abb-ba7d-26be66ae8a29 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:04.669035 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceeaf49-480e-4518-ae59-6d2043541bb1 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:04.704536 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:04.706932 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d96f7f-3545-416c-866c-17c84b849ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:04.707568 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d96f7f-3545-416c-866c-17c84b849ae5 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:04.707844 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d96f7f-3545-416c-866c-17c84b849ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:04.708114 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d96f7f-3545-416c-866c-17c84b849ae5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:04.715528 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d96f7f-3545-416c-866c-17c84b849ae5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:04.717407 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d96f7f-3545-416c-866c-17c84b849ae5 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:04.729654 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-e7d96f7f-3545-416c-866c-17c84b849ae5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:04.729654 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:04.729654 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:04.729654 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:04.729654 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:04.729654 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:04.729654 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:04.729654 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:04.729654 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:04.729654 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:04.729654 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:04.729654 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:04.729654 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:29:04.729654 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:29:04.729654 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:29:04.730526 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:29:04.730526 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:04.730526 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:04.730526 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:04.730526 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:04.730526 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:04.730526 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:04.730526 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:04.730526 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:04.730526 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:04.730526 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:04.730526 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:04.730526 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:04.730526 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:04.730526 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:04.730526 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:04.730526 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:04.731374 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:29:04.731374 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:29:04.731374 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:29:04.731374 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:29:04.731374 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:29:04.731374 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:29:04.731374 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:04.731374 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:04.731374 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 275/529] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:29:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:04.731845 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d96f7f-3545-416c-866c-17c84b849ae5 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:04.743443 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c97755-7c74-4abb-ba7d-26be66ae8a29 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:04.779893 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba600fb3-bfc3-4a8e-b407-a45bd1137478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:04.780502 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba600fb3-bfc3-4a8e-b407-a45bd1137478 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:04.780800 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba600fb3-bfc3-4a8e-b407-a45bd1137478 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:04.781076 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba600fb3-bfc3-4a8e-b407-a45bd1137478 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:04.781793 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba600fb3-bfc3-4a8e-b407-a45bd1137478 None admin] RBAC: Adding query filter params (role.id=db83c2f8e21945c3ab9642c620d29a23, scope.project.id=ee84c3340f2641ad963655a7a212ef96, user.id=9c2569e7cd774199aa5750fd6a850060) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:04.782101 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba600fb3-bfc3-4a8e-b407-a45bd1137478 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:04.784159 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba600fb3-bfc3-4a8e-b407-a45bd1137478 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:04.796746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 255/530] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:29:04 2022] GET /identity/v3/role_assignments?user.id=9c2569e7cd774199aa5750fd6a850060&scope.project.id=ee84c3340f2641ad963655a7a212ef96&role.id=db83c2f8e21945c3ab9642c620d29a23 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:04.796746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba600fb3-bfc3-4a8e-b407-a45bd1137478 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:04.855798 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:04.867135 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d96f7f-3545-416c-866c-17c84b849ae5 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:04.908055 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd09b1-3f6c-4587-9984-d67fe0079727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:04.909465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd09b1-3f6c-4587-9984-d67fe0079727 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:04.909841 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd09b1-3f6c-4587-9984-d67fe0079727 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:04.910120 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd09b1-3f6c-4587-9984-d67fe0079727 None admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:04.910871 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd09b1-3f6c-4587-9984-d67fe0079727 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:04.911247 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd09b1-3f6c-4587-9984-d67fe0079727 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:04.912812 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd09b1-3f6c-4587-9984-d67fe0079727 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:04.938858 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 276/531] 10.222.0.52 () {56 vars in 1139 bytes} [Mon Aug 29 22:29:04 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:04.940617 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd09b1-3f6c-4587-9984-d67fe0079727 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:04.951139 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba600fb3-bfc3-4a8e-b407-a45bd1137478 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:04.988777 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc7a06-b807-4564-944b-e418ba6c17fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:04.989347 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc7a06-b807-4564-944b-e418ba6c17fa None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:04.989629 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc7a06-b807-4564-944b-e418ba6c17fa None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:04.989904 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc7a06-b807-4564-944b-e418ba6c17fa None admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:04.990602 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cc7a06-b807-4564-944b-e418ba6c17fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:04.990912 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cc7a06-b807-4564-944b-e418ba6c17fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:04.992409 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cc7a06-b807-4564-944b-e418ba6c17fa None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:04.995060 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:05.007832 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 256/532] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Aug 29 22:29:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:05.008673 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc7a06-b807-4564-944b-e418ba6c17fa None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:05.068988 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd09b1-3f6c-4587-9984-d67fe0079727 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:05.117824 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48aabbf-bdda-4ff4-bb9e-200b7818e056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:05.118354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48aabbf-bdda-4ff4-bb9e-200b7818e056 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:05.118640 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48aabbf-bdda-4ff4-bb9e-200b7818e056 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:05.118904 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc7a06-b807-4564-944b-e418ba6c17fa None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:05.119039 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48aabbf-bdda-4ff4-bb9e-200b7818e056 None admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:05.119602 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48aabbf-bdda-4ff4-bb9e-200b7818e056 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:05.119907 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48aabbf-bdda-4ff4-bb9e-200b7818e056 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:05.122787 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48aabbf-bdda-4ff4-bb9e-200b7818e056 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:05.151873 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 277/533] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Aug 29 22:29:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:05.152875 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48aabbf-bdda-4ff4-bb9e-200b7818e056 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:05.161881 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37fccac-d4a1-4a6f-af3a-fe939fb0f528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:05.162651 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37fccac-d4a1-4a6f-af3a-fe939fb0f528 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:05.162956 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37fccac-d4a1-4a6f-af3a-fe939fb0f528 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:05.163328 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37fccac-d4a1-4a6f-af3a-fe939fb0f528 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:05.165780 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:05.248526 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:05.255506 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37fccac-d4a1-4a6f-af3a-fe939fb0f528 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:05.257111 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37fccac-d4a1-4a6f-af3a-fe939fb0f528 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:05.291270 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48aabbf-bdda-4ff4-bb9e-200b7818e056 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:05.338712 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e65b09-4ad2-42e3-bed0-f7b911d4caa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:05.339589 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e65b09-4ad2-42e3-bed0-f7b911d4caa2 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:05.340085 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e65b09-4ad2-42e3-bed0-f7b911d4caa2 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:05.340509 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e65b09-4ad2-42e3-bed0-f7b911d4caa2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:05.341409 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e65b09-4ad2-42e3-bed0-f7b911d4caa2 None admin] RBAC: Adding query filter params (role.id=db83c2f8e21945c3ab9642c620d29a23, scope.project.id=ee84c3340f2641ad963655a7a212ef96, user.id=bdbf961e02934e88af3a72d6218f32fa) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:05.341854 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e65b09-4ad2-42e3-bed0-f7b911d4caa2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:05.343942 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e65b09-4ad2-42e3-bed0-f7b911d4caa2 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:05.360911 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 278/534] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:29:05 2022] GET /identity/v3/role_assignments?user.id=bdbf961e02934e88af3a72d6218f32fa&scope.project.id=ee84c3340f2641ad963655a7a212ef96&role.id=db83c2f8e21945c3ab9642c620d29a23 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:05.360911 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e65b09-4ad2-42e3-bed0-f7b911d4caa2 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:05.394406 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 257/535] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:29:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:29:05.398673 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37fccac-d4a1-4a6f-af3a-fe939fb0f528 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:05.513301 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:05.568922 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e65b09-4ad2-42e3-bed0-f7b911d4caa2 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:05.577009 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:05.583010 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37fccac-d4a1-4a6f-af3a-fe939fb0f528 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:05.615189 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aec62b-5283-40c3-b37a-29f6baef9ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:05.616434 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aec62b-5283-40c3-b37a-29f6baef9ade None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:05.617291 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aec62b-5283-40c3-b37a-29f6baef9ade None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:05.617794 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aec62b-5283-40c3-b37a-29f6baef9ade None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:05.619378 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aec62b-5283-40c3-b37a-29f6baef9ade None admin] RBAC: Adding query filter params (role.id=db83c2f8e21945c3ab9642c620d29a23, scope.project.id=ee84c3340f2641ad963655a7a212ef96, user.id=6e782c8acaaa42c4bd78ddc7c8cc654b) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:05.619944 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aec62b-5283-40c3-b37a-29f6baef9ade None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:05.621656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aec62b-5283-40c3-b37a-29f6baef9ade None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:05.623117 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e357c619-be28-4716-b9aa-0014256bb28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:05.623876 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e357c619-be28-4716-b9aa-0014256bb28e None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:05.624255 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e357c619-be28-4716-b9aa-0014256bb28e None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:05.624630 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e357c619-be28-4716-b9aa-0014256bb28e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:05.629770 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 279/536] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:29:05 2022] GET /identity/v3/role_assignments?user.id=6e782c8acaaa42c4bd78ddc7c8cc654b&scope.project.id=ee84c3340f2641ad963655a7a212ef96&role.id=db83c2f8e21945c3ab9642c620d29a23 => generated 259 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:05.650915 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e357c619-be28-4716-b9aa-0014256bb28e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:05.652402 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e357c619-be28-4716-b9aa-0014256bb28e None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:05.683164 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-e357c619-be28-4716-b9aa-0014256bb28e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:05.683164 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:05.683164 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:05.683164 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:05.683164 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:05.683164 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:05.683164 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:05.683164 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:05.683164 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:05.683164 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:05.683164 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:05.683164 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:05.683164 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:05.683164 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:05.683164 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:05.683164 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:05.684320 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:05.684320 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:05.684320 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:05.684320 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:05.684320 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:05.684320 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:05.684320 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:05.684320 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:05.684320 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:05.684320 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:05.684320 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:05.684320 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:05.684320 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:05.684320 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:05.684320 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:05.684320 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:05.684320 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:05.685417 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:05.685417 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:05.685417 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:05.685417 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 258/537] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:05.690710 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aec62b-5283-40c3-b37a-29f6baef9ade None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:05.853507 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aec62b-5283-40c3-b37a-29f6baef9ade None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:05.906394 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f8313-53df-491f-a79c-fa95f3d85ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:05.906922 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f8313-53df-491f-a79c-fa95f3d85ec3 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:05.907200 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f8313-53df-491f-a79c-fa95f3d85ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:05.907467 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f8313-53df-491f-a79c-fa95f3d85ec3 None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:05.908111 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0f8313-53df-491f-a79c-fa95f3d85ec3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:05.908443 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0f8313-53df-491f-a79c-fa95f3d85ec3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:05.909936 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0f8313-53df-491f-a79c-fa95f3d85ec3 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:05.914924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:05.927113 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 280/538] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:05.932365 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e357c619-be28-4716-b9aa-0014256bb28e None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:06.098854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e357c619-be28-4716-b9aa-0014256bb28e None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:06.146336 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed501a-cbf4-4140-9b87-ecb36e37ba89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:06.146976 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed501a-cbf4-4140-9b87-ecb36e37ba89 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:06.147260 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed501a-cbf4-4140-9b87-ecb36e37ba89 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:06.147538 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed501a-cbf4-4140-9b87-ecb36e37ba89 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:06.157851 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ed501a-cbf4-4140-9b87-ecb36e37ba89 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:06.159850 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ed501a-cbf4-4140-9b87-ecb36e37ba89 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:06.162116 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b95847-3110-405b-8ac4-ccb97a0dc3fd None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:06.162406 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b95847-3110-405b-8ac4-ccb97a0dc3fd None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:06.162683 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b95847-3110-405b-8ac4-ccb97a0dc3fd None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:06.164256 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 281/539] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:29:06.169204 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa033e-baee-4f36-9d67-df2c0a4bffcc None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:06.169752 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa033e-baee-4f36-9d67-df2c0a4bffcc None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:06.169752 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa033e-baee-4f36-9d67-df2c0a4bffcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:06.177917 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-50ed501a-cbf4-4140-9b87-ecb36e37ba89 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:06.177917 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:06.177917 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:06.177917 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:06.177917 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:06.177917 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:06.177917 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:06.177917 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:06.177917 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:06.177917 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:06.177917 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:06.177917 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:06.177917 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:29:06.177917 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:29:06.177917 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:29:06.178801 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:29:06.178801 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:06.178801 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:06.178801 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:06.178801 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:06.178801 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:06.178801 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:06.178801 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:06.178801 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:06.178801 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:06.178801 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:06.178801 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:06.178801 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:06.178801 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:06.178801 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:06.178801 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:06.178801 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:06.179622 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:29:06.179622 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:29:06.179622 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:29:06.179622 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:29:06.179622 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:29:06.179622 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:29:06.179622 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:06.179622 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:06.179622 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 259/540] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:29:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:06.185367 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed501a-cbf4-4140-9b87-ecb36e37ba89 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:06.194563 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-55fa033e-baee-4f36-9d67-df2c0a4bffcc None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:06.201584 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-55fa033e-baee-4f36-9d67-df2c0a4bffcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:06.206940 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:06.337768 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 282/541] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 22:29:06.366532 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed501a-cbf4-4140-9b87-ecb36e37ba89 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:06.403134 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e1586-1c31-47cf-979e-9725f8cf5ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:06.404008 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e1586-1c31-47cf-979e-9725f8cf5ca6 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:06.404447 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e1586-1c31-47cf-979e-9725f8cf5ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:06.404897 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e1586-1c31-47cf-979e-9725f8cf5ca6 None admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:06.405723 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e1586-1c31-47cf-979e-9725f8cf5ca6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:06.406172 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e1586-1c31-47cf-979e-9725f8cf5ca6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:06.407764 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e1586-1c31-47cf-979e-9725f8cf5ca6 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:06.427529 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 260/542] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Aug 29 22:29:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:29:06.429114 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa033e-baee-4f36-9d67-df2c0a4bffcc None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:06.566951 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa033e-baee-4f36-9d67-df2c0a4bffcc None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:06.601635 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:06.617181 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028697a8-8706-48e3-93a0-121ca9c161f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:06.620346 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028697a8-8706-48e3-93a0-121ca9c161f2 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:06.620346 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028697a8-8706-48e3-93a0-121ca9c161f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:06.620346 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028697a8-8706-48e3-93a0-121ca9c161f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:06.620346 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028697a8-8706-48e3-93a0-121ca9c161f2 None admin] RBAC: Adding query filter params (role.id=db83c2f8e21945c3ab9642c620d29a23, scope.project.id=ee84c3340f2641ad963655a7a212ef96, user.id=0fe5e8c4cb324ffb9d5b80f18d123727) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:06.620346 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028697a8-8706-48e3-93a0-121ca9c161f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:06.621696 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028697a8-8706-48e3-93a0-121ca9c161f2 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:06.632899 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 283/543] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:29:06 2022] GET /identity/v3/role_assignments?user.id=0fe5e8c4cb324ffb9d5b80f18d123727&scope.project.id=ee84c3340f2641ad963655a7a212ef96&role.id=db83c2f8e21945c3ab9642c620d29a23 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:06.765667 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e1586-1c31-47cf-979e-9725f8cf5ca6 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:06.840034 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:06.878052 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e1586-1c31-47cf-979e-9725f8cf5ca6 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:06.917425 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adb2fc7-71bc-4c76-9db4-bd93abe903ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:06.918073 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb2fc7-71bc-4c76-9db4-bd93abe903ea None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:06.918357 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb2fc7-71bc-4c76-9db4-bd93abe903ea None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:06.918634 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb2fc7-71bc-4c76-9db4-bd93abe903ea None admin] PATH_INFO: `/v3/roles/service` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:06.932193 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adb2fc7-71bc-4c76-9db4-bd93abe903ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:29:06.937222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-6adb2fc7-71bc-4c76-9db4-bd93abe903ea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:29:06.937222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:06.937222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:06.937222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:06.937222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:06.937222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:06.937222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:06.937222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:06.937222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:06.937222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:06.937222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:06.937222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:06.937222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:29:06.937222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:29:06.937222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:29:06.937222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:29:06.938151 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:29:06.938151 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:29:06.938151 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:29:06.938151 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:29:06.938151 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:06.938151 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:06.938151 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:06.938151 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:06.938151 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:06.938151 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:06.938151 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:06.938151 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:06.938151 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:06.938151 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:06.938151 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:06.938151 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:06.938950 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:06.938950 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:06.938950 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:06.938950 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:06.938950 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:29:06.938950 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:29:06.938950 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:29:06.938950 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:29:06.938950 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:29:06.938950 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:29:06.938950 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:29:06.938950 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:06.944890 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 261/544] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Aug 29 22:29:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:06.946142 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028697a8-8706-48e3-93a0-121ca9c161f2 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:06.999425 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb56f2-a62b-415d-b270-e9f828435040 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:06.999772 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb56f2-a62b-415d-b270-e9f828435040 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:06.999977 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb56f2-a62b-415d-b270-e9f828435040 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:07.001810 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 262/545] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:29:07.007479 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ffd6a-36d4-4976-894d-a5fd8c602089 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:07.007758 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ffd6a-36d4-4976-894d-a5fd8c602089 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:07.008029 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ffd6a-36d4-4976-894d-a5fd8c602089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:07.027854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-433ffd6a-36d4-4976-894d-a5fd8c602089 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:07.028997 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-433ffd6a-36d4-4976-894d-a5fd8c602089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:07.111155 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028697a8-8706-48e3-93a0-121ca9c161f2 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:07.155815 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 263/546] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 22:29:07.156313 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af7a075-f56b-4055-81b0-862019cbd5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:07.157006 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af7a075-f56b-4055-81b0-862019cbd5a1 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:07.157289 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af7a075-f56b-4055-81b0-862019cbd5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:07.157614 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af7a075-f56b-4055-81b0-862019cbd5a1 None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:07.158842 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af7a075-f56b-4055-81b0-862019cbd5a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:07.159147 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af7a075-f56b-4055-81b0-862019cbd5a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:07.160561 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af7a075-f56b-4055-81b0-862019cbd5a1 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:07.169718 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 284/547] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Aug 29 22:29:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:07.176769 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433ffd6a-36d4-4976-894d-a5fd8c602089 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:07.258686 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:07.310673 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433ffd6a-36d4-4976-894d-a5fd8c602089 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:07.360384 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3632c493-eab3-4f1c-86f7-ac979d43f879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:07.361061 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632c493-eab3-4f1c-86f7-ac979d43f879 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:07.361399 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632c493-eab3-4f1c-86f7-ac979d43f879 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:07.361735 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632c493-eab3-4f1c-86f7-ac979d43f879 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:07.373400 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3632c493-eab3-4f1c-86f7-ac979d43f879 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:07.374867 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3632c493-eab3-4f1c-86f7-ac979d43f879 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:07.395106 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-3632c493-eab3-4f1c-86f7-ac979d43f879 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:07.395106 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:07.395106 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:07.395106 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:07.395106 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:07.395106 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:07.395106 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:07.395106 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:07.395106 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:07.395106 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:07.395106 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:07.395106 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:07.395106 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:07.395106 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:07.395106 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:07.395106 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:07.396061 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:07.396061 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:07.396061 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:07.396061 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:07.396061 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:07.396061 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:07.396061 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:07.396061 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:07.396061 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:07.396061 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:07.396061 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:07.396061 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:07.396061 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:07.396061 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:07.396061 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:07.396061 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:07.396061 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:07.397432 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:07.397432 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:07.397432 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:07.401014 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 264/548] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:07.409837 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af7a075-f56b-4055-81b0-862019cbd5a1 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:07.503671 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:07.592645 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af7a075-f56b-4055-81b0-862019cbd5a1 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:07.637327 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e9333-7b41-4ce2-bcfe-3d2bf33e98c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:07.637930 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e9333-7b41-4ce2-bcfe-3d2bf33e98c6 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:07.638206 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e9333-7b41-4ce2-bcfe-3d2bf33e98c6 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:07.638478 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e9333-7b41-4ce2-bcfe-3d2bf33e98c6 None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:07.639124 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6e9333-7b41-4ce2-bcfe-3d2bf33e98c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:07.639425 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6e9333-7b41-4ce2-bcfe-3d2bf33e98c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:07.640805 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6e9333-7b41-4ce2-bcfe-3d2bf33e98c6 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:07.662667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 285/549] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:07.668135 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3632c493-eab3-4f1c-86f7-ac979d43f879 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:07.681032 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e9333-7b41-4ce2-bcfe-3d2bf33e98c6 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:07.792090 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3632c493-eab3-4f1c-86f7-ac979d43f879 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:07.803812 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:07.829675 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63532872-2c77-496e-916e-09f941cd3ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:07.830394 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63532872-2c77-496e-916e-09f941cd3ae2 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:07.830711 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63532872-2c77-496e-916e-09f941cd3ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:07.831056 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63532872-2c77-496e-916e-09f941cd3ae2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:07.839818 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63532872-2c77-496e-916e-09f941cd3ae2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:07.841346 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63532872-2c77-496e-916e-09f941cd3ae2 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:07.847092 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e9333-7b41-4ce2-bcfe-3d2bf33e98c6 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:07.856091 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-63532872-2c77-496e-916e-09f941cd3ae2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 22:29:07.856091 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:07.856091 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:07.856091 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:07.856091 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:07.856091 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:07.856091 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:07.856091 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:07.856091 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:07.856091 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:07.856091 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:07.856091 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:07.856091 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:29:07.856091 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:29:07.856091 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:29:07.856091 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:29:07.857046 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:07.857046 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:07.857046 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:29:07.857046 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:07.857046 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:29:07.857046 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:07.857046 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:07.857046 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:07.857046 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:07.857046 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:07.857046 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:07.857046 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:07.857046 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:07.857046 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:07.857046 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:07.857046 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:07.857046 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:07.857046 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:07.858554 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:07.858554 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:07.858554 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:29:07.858554 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:29:07.858554 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:29:07.858554 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:29:07.858554 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:29:07.858554 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:29:07.858554 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 22:29:07.858554 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:07.858554 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 265/550] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Aug 29 22:29:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:07.859133 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58034a5-eb48-4961-ade9-64daee3db3ec None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:07.859133 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58034a5-eb48-4961-ade9-64daee3db3ec None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:07.859422 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58034a5-eb48-4961-ade9-64daee3db3ec None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:07.860794 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 266/551] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:29:07.863087 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58034a5-eb48-4961-ade9-64daee3db3ec None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:07.866129 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:07.894702 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ff8173-ef90-432f-997a-620e6680f4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:07.895682 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ff8173-ef90-432f-997a-620e6680f4f0 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:07.896270 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ff8173-ef90-432f-997a-620e6680f4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:07.896908 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ff8173-ef90-432f-997a-620e6680f4f0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:07.909998 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ff8173-ef90-432f-997a-620e6680f4f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:29:07.915566 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-76ff8173-ef90-432f-997a-620e6680f4f0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:29:07.915566 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:07.915566 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:07.915566 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:07.915566 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:07.915566 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:07.915566 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:07.915566 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:07.915566 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:07.915566 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:07.915566 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:07.915566 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:07.915566 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:29:07.915566 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:29:07.915566 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:29:07.915566 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:29:07.916463 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:29:07.916463 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:29:07.916463 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:29:07.916463 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:29:07.916463 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:07.916463 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:07.916463 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:07.916463 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:07.916463 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:07.916463 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:07.916463 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:07.916463 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:07.916463 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:07.916463 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:07.916463 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:07.916463 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:07.917310 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:07.917310 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:07.917310 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:07.917310 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:07.917310 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:29:07.917310 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:29:07.917310 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:29:07.917310 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:29:07.917310 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:29:07.917310 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:29:07.917310 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:29:07.917310 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:07.920046 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 286/552] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Aug 29 22:29:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:29:07.921489 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac1f04-6788-430f-b292-a5f2e3905beb None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:07.921972 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac1f04-6788-430f-b292-a5f2e3905beb None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:07.922450 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac1f04-6788-430f-b292-a5f2e3905beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:07.944331 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-aeac1f04-6788-430f-b292-a5f2e3905beb None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:07.945944 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-aeac1f04-6788-430f-b292-a5f2e3905beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:07.975355 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58034a5-eb48-4961-ade9-64daee3db3ec None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:08.016370 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ae5fbf-62b9-4750-b9d4-9b3305f18a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:08.017009 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae5fbf-62b9-4750-b9d4-9b3305f18a99 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:08.017291 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae5fbf-62b9-4750-b9d4-9b3305f18a99 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:08.017618 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae5fbf-62b9-4750-b9d4-9b3305f18a99 None admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:08.018454 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ae5fbf-62b9-4750-b9d4-9b3305f18a99 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:08.018751 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ae5fbf-62b9-4750-b9d4-9b3305f18a99 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:08.020222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ae5fbf-62b9-4750-b9d4-9b3305f18a99 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:08.034492 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 287/553] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:29:08.034492 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac1f04-6788-430f-b292-a5f2e3905beb None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:08.044573 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 267/554] 10.222.0.52 () {56 vars in 1135 bytes} [Mon Aug 29 22:29:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:08.046262 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d980802-14f1-4f20-a12a-004c88c40f1f None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:08.046554 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d980802-14f1-4f20-a12a-004c88c40f1f None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:08.046838 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d980802-14f1-4f20-a12a-004c88c40f1f None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:08.048551 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 268/555] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:29:08.050361 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d980802-14f1-4f20-a12a-004c88c40f1f None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:08.081840 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:08.171976 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d980802-14f1-4f20-a12a-004c88c40f1f None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:08.177081 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac1f04-6788-430f-b292-a5f2e3905beb None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:08.178629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:08.209980 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643bdd6-71d4-4704-b1fc-d0e0b3ed484b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:08.210590 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643bdd6-71d4-4704-b1fc-d0e0b3ed484b None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:08.210870 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643bdd6-71d4-4704-b1fc-d0e0b3ed484b None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:08.211140 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643bdd6-71d4-4704-b1fc-d0e0b3ed484b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:08.222038 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faeba69-3e22-4e20-ba42-84078ce39478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:08.222647 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faeba69-3e22-4e20-ba42-84078ce39478 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:08.223404 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faeba69-3e22-4e20-ba42-84078ce39478 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:08.223678 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faeba69-3e22-4e20-ba42-84078ce39478 None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:08.224346 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faeba69-3e22-4e20-ba42-84078ce39478 None admin] RBAC: Adding query filter params (name=service) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:08.225238 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faeba69-3e22-4e20-ba42-84078ce39478 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:08.226939 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faeba69-3e22-4e20-ba42-84078ce39478 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:08.238372 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 288/556] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Aug 29 22:29:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:08.240361 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882cb038-67e5-42d5-bd3b-9a53f28cd971 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:08.240650 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882cb038-67e5-42d5-bd3b-9a53f28cd971 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:08.240948 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882cb038-67e5-42d5-bd3b-9a53f28cd971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:08.263762 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-882cb038-67e5-42d5-bd3b-9a53f28cd971 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:08.265145 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-882cb038-67e5-42d5-bd3b-9a53f28cd971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:08.296482 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8643bdd6-71d4-4704-b1fc-d0e0b3ed484b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:08.298318 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8643bdd6-71d4-4704-b1fc-d0e0b3ed484b None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:08.329581 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:08.345354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 289/557] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:29:08.345354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882cb038-67e5-42d5-bd3b-9a53f28cd971 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:08.420248 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 269/558] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:29:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:29:08.421195 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643bdd6-71d4-4704-b1fc-d0e0b3ed484b None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:08.451464 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:08.459815 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882cb038-67e5-42d5-bd3b-9a53f28cd971 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:08.498364 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21cead6-9402-4049-b6e3-38b20ada438b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:08.499344 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21cead6-9402-4049-b6e3-38b20ada438b None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:08.499920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21cead6-9402-4049-b6e3-38b20ada438b None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:08.500438 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21cead6-9402-4049-b6e3-38b20ada438b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:08.512116 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21cead6-9402-4049-b6e3-38b20ada438b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:08.514053 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21cead6-9402-4049-b6e3-38b20ada438b None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:08.530565 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-c21cead6-9402-4049-b6e3-38b20ada438b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:08.530565 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:08.530565 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:08.530565 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:08.530565 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:08.530565 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:08.530565 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:08.530565 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:08.530565 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:08.530565 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:08.530565 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:08.530565 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:08.530565 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:08.530565 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:08.530565 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:08.530565 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:08.531744 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:08.531744 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:08.531744 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:08.531744 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:08.531744 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:08.531744 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:08.531744 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:08.531744 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:08.531744 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:08.531744 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:08.531744 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:08.531744 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:08.531744 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:08.531744 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:08.531744 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:08.531744 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:08.531744 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:08.532879 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:08.532879 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:08.532879 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:08.535113 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 290/559] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:08.536128 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21cead6-9402-4049-b6e3-38b20ada438b None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:08.544188 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643bdd6-71d4-4704-b1fc-d0e0b3ed484b None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:08.576203 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:08.585890 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b1a83-02ca-4650-8a53-fea7583a6e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:08.590395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b1a83-02ca-4650-8a53-fea7583a6e22 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:08.590395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b1a83-02ca-4650-8a53-fea7583a6e22 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:08.590395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b1a83-02ca-4650-8a53-fea7583a6e22 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:08.599240 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300b1a83-02ca-4650-8a53-fea7583a6e22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:29:08.604532 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-300b1a83-02ca-4650-8a53-fea7583a6e22 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:29:08.604532 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:08.604532 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:08.604532 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:08.604532 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:08.604532 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:08.604532 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:08.604532 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:08.604532 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:08.604532 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:08.604532 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:08.604532 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:08.604532 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:29:08.604532 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:29:08.604532 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:29:08.604532 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:29:08.605711 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:29:08.605711 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:29:08.605711 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:29:08.605711 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:29:08.605711 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:08.605711 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:08.605711 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:08.605711 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:08.605711 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:08.605711 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:08.605711 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:08.605711 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:08.605711 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:08.605711 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:08.605711 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:08.605711 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:08.606950 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:08.606950 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:08.606950 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:08.606950 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:08.606950 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:29:08.606950 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:29:08.606950 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:29:08.606950 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:29:08.606950 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:29:08.606950 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:29:08.606950 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:29:08.606950 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:08.606950 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 270/560] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Aug 29 22:29:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:08.607658 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b1a83-02ca-4650-8a53-fea7583a6e22 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:08.656123 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21cead6-9402-4049-b6e3-38b20ada438b None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:08.695255 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419870b4-0457-4d6b-bf63-8b77f7263a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:08.695923 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419870b4-0457-4d6b-bf63-8b77f7263a4e None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:08.696234 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419870b4-0457-4d6b-bf63-8b77f7263a4e None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:08.696534 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419870b4-0457-4d6b-bf63-8b77f7263a4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:08.709395 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419870b4-0457-4d6b-bf63-8b77f7263a4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:08.710859 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419870b4-0457-4d6b-bf63-8b77f7263a4e None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:08.717973 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:08.718400 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b1a83-02ca-4650-8a53-fea7583a6e22 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:08.726609 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-419870b4-0457-4d6b-bf63-8b77f7263a4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:08.726609 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:08.726609 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:08.726609 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:08.726609 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:08.726609 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:08.726609 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:08.726609 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:08.726609 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:08.726609 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:08.726609 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:08.726609 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:08.726609 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:08.726609 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:08.726609 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:08.726609 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:08.733698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:08.733698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:08.733698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:08.733698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:08.733698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:08.733698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:08.733698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:08.733698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:08.733698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:08.733698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:08.733698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:08.733698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:08.733698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:08.733698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:08.733698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:08.733698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:08.733698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:08.734783 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:08.734783 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:08.734783 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:08.734783 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 291/561] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:08.734783 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419870b4-0457-4d6b-bf63-8b77f7263a4e None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:08.760500 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b48d21-127e-47f5-8fd3-709e1dd182b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:08.761132 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b48d21-127e-47f5-8fd3-709e1dd182b8 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:08.761431 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b48d21-127e-47f5-8fd3-709e1dd182b8 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:08.761686 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b48d21-127e-47f5-8fd3-709e1dd182b8 None admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:08.762329 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b48d21-127e-47f5-8fd3-709e1dd182b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:08.762633 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b48d21-127e-47f5-8fd3-709e1dd182b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:08.764103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b48d21-127e-47f5-8fd3-709e1dd182b8 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:08.782055 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 271/562] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:08.782055 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b48d21-127e-47f5-8fd3-709e1dd182b8 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:08.841735 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:08.854340 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419870b4-0457-4d6b-bf63-8b77f7263a4e None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:08.891166 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d71acf1-d3dd-4c6e-8a2f-ead15d62ed12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:08.891769 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71acf1-d3dd-4c6e-8a2f-ead15d62ed12 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:08.892077 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71acf1-d3dd-4c6e-8a2f-ead15d62ed12 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:08.892335 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71acf1-d3dd-4c6e-8a2f-ead15d62ed12 None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:08.894056 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d71acf1-d3dd-4c6e-8a2f-ead15d62ed12 None admin] RBAC: Adding query filter params (name=service) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:08.894391 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d71acf1-d3dd-4c6e-8a2f-ead15d62ed12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:08.895781 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d71acf1-d3dd-4c6e-8a2f-ead15d62ed12 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:08.896950 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b48d21-127e-47f5-8fd3-709e1dd182b8 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:08.908955 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 292/563] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Aug 29 22:29:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:08.908955 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d71acf1-d3dd-4c6e-8a2f-ead15d62ed12 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:08.956156 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a3c73-e9e7-4bd9-9041-4908f804f6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:08.956809 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a3c73-e9e7-4bd9-9041-4908f804f6b9 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:08.957091 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a3c73-e9e7-4bd9-9041-4908f804f6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:08.957560 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a3c73-e9e7-4bd9-9041-4908f804f6b9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:08.967755 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5a3c73-e9e7-4bd9-9041-4908f804f6b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:29:08.973182 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-ef5a3c73-e9e7-4bd9-9041-4908f804f6b9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:29:08.973182 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:08.973182 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:08.973182 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:08.973182 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:08.973182 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:08.973182 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:08.973182 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:08.973182 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:08.973182 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:08.973182 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:08.973182 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:08.973182 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:29:08.973182 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:29:08.973182 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:29:08.973182 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:29:08.974188 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:29:08.974188 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:29:08.974188 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:29:08.974188 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:29:08.974188 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:08.974188 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:08.974188 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:08.974188 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:08.974188 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:08.974188 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:08.974188 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:08.974188 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:08.974188 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:08.974188 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:08.974188 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:08.974188 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:08.975164 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:08.975164 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:08.975164 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:08.975164 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:08.975164 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:29:08.975164 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:29:08.975164 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:29:08.975164 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:29:08.975164 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:29:08.975164 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:29:08.975164 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:29:08.975164 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:08.975164 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 272/564] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Aug 29 22:29:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:08.975164 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a3c73-e9e7-4bd9-9041-4908f804f6b9 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:08.990119 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:09.042063 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d71acf1-d3dd-4c6e-8a2f-ead15d62ed12 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:09.076780 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c0a39-e1cf-46c8-ac42-9828505d550c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:09.077346 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c0a39-e1cf-46c8-ac42-9828505d550c None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:09.077623 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c0a39-e1cf-46c8-ac42-9828505d550c None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:09.077899 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c0a39-e1cf-46c8-ac42-9828505d550c None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:09.078537 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4c0a39-e1cf-46c8-ac42-9828505d550c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:09.078842 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4c0a39-e1cf-46c8-ac42-9828505d550c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:09.080304 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4c0a39-e1cf-46c8-ac42-9828505d550c None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:09.099920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 293/565] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:09.099920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c0a39-e1cf-46c8-ac42-9828505d550c None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:09.111879 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a3c73-e9e7-4bd9-9041-4908f804f6b9 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:09.120931 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:09.146403 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14851c45-53fa-4095-961d-3a5c83aa915a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:09.147028 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14851c45-53fa-4095-961d-3a5c83aa915a None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:09.158097 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14851c45-53fa-4095-961d-3a5c83aa915a None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:09.158097 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14851c45-53fa-4095-961d-3a5c83aa915a None admin] PATH_INFO: `/v3/users/glance` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:09.158097 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14851c45-53fa-4095-961d-3a5c83aa915a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:09.158097 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14851c45-53fa-4095-961d-3a5c83aa915a None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:09.169174 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-14851c45-53fa-4095-961d-3a5c83aa915a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 29 22:29:09.169174 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:09.169174 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:09.169174 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:09.169174 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:09.169174 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:09.169174 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:09.169174 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:09.169174 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:09.169174 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:09.169174 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:09.169174 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:09.169174 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:29:09.169174 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:29:09.169174 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:29:09.169174 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:29:09.170427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:09.170427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:09.170427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:29:09.170427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:09.170427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:29:09.170427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:09.170427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:09.170427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:09.170427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:09.170427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:09.170427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:09.170427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:09.170427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:09.170427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:09.170427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:09.170427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:09.170427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:09.170427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:09.171861 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:09.171861 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:09.171861 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:29:09.171861 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:29:09.171861 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:29:09.171861 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:29:09.171861 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:29:09.171861 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:29:09.171861 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 29 22:29:09.171861 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:09.171861 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 273/566] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:29:08 2022] GET /identity/v3/users/glance => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:09.171861 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3432c-89d1-469a-94cb-8d37ce450a0b None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:09.172656 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3432c-89d1-469a-94cb-8d37ce450a0b None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:09.172656 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3432c-89d1-469a-94cb-8d37ce450a0b None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:09.174333 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 274/567] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:29:09.175028 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3432c-89d1-469a-94cb-8d37ce450a0b None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:09.206718 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c0a39-e1cf-46c8-ac42-9828505d550c None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:09.227329 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:09.241297 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03e76d8-0b2a-48a4-b8f2-112f3dbed190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:09.241923 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e76d8-0b2a-48a4-b8f2-112f3dbed190 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:09.243757 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e76d8-0b2a-48a4-b8f2-112f3dbed190 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:09.243757 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e76d8-0b2a-48a4-b8f2-112f3dbed190 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:09.252678 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03e76d8-0b2a-48a4-b8f2-112f3dbed190 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:09.254750 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03e76d8-0b2a-48a4-b8f2-112f3dbed190 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:09.270059 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-d03e76d8-0b2a-48a4-b8f2-112f3dbed190 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:09.270059 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:09.270059 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:09.270059 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:09.270059 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:09.270059 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:09.270059 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:09.270059 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:09.270059 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:09.270059 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:09.270059 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:09.270059 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:09.270059 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:09.270059 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:09.270059 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:09.270059 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:09.271024 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:09.271024 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:09.271024 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:09.271024 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:09.271024 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:09.271024 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:09.271024 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:09.271024 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:09.271024 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:09.271024 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:09.271024 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:09.271024 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:09.271024 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:09.271024 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:09.271024 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:09.271024 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:09.271024 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:09.271949 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:09.271949 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:09.271949 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:09.271949 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 294/568] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:09.271949 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03e76d8-0b2a-48a4-b8f2-112f3dbed190 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:09.281764 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3432c-89d1-469a-94cb-8d37ce450a0b None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:09.316741 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546f8001-83a0-4933-a6a3-0c39471dcf6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:09.317348 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f8001-83a0-4933-a6a3-0c39471dcf6a None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:09.317624 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f8001-83a0-4933-a6a3-0c39471dcf6a None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:09.317892 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f8001-83a0-4933-a6a3-0c39471dcf6a None admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:09.318587 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546f8001-83a0-4933-a6a3-0c39471dcf6a None admin] RBAC: Adding query filter params (name=service) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:09.318910 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546f8001-83a0-4933-a6a3-0c39471dcf6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:09.320335 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546f8001-83a0-4933-a6a3-0c39471dcf6a None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:09.328748 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 275/569] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Aug 29 22:29:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:09.329486 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546f8001-83a0-4933-a6a3-0c39471dcf6a None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:09.362095 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:09.380749 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03e76d8-0b2a-48a4-b8f2-112f3dbed190 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:09.416050 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0f5eb6-07a5-4de6-af9f-20e3ba565e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:09.416674 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0f5eb6-07a5-4de6-af9f-20e3ba565e92 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:09.416986 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0f5eb6-07a5-4de6-af9f-20e3ba565e92 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:09.417261 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0f5eb6-07a5-4de6-af9f-20e3ba565e92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:09.424313 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0f5eb6-07a5-4de6-af9f-20e3ba565e92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:09.425905 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0f5eb6-07a5-4de6-af9f-20e3ba565e92 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:09.433110 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546f8001-83a0-4933-a6a3-0c39471dcf6a None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:09.437067 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-6d0f5eb6-07a5-4de6-af9f-20e3ba565e92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:09.437067 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:09.437067 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:09.437067 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:09.437067 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:09.437067 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:09.437067 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:09.437067 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:09.437067 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:09.437067 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:09.437067 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:09.437067 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:09.437067 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:29:09.437067 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:29:09.437067 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:29:09.438094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:29:09.438094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:09.438094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:09.438094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:09.438094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:09.438094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:09.438094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:09.438094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:09.438094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:09.438094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:09.438094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:09.438094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:09.438094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:09.438094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:09.438094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:09.438094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:09.438094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:09.439337 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:29:09.439337 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:29:09.439337 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:29:09.439337 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:29:09.439337 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:29:09.439337 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:29:09.439337 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:09.439337 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:09.439337 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 295/570] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:29:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:09.440337 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5e209-a5fd-4de2-86a4-82633b2c8687 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:09.440663 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5e209-a5fd-4de2-86a4-82633b2c8687 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:09.441003 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5e209-a5fd-4de2-86a4-82633b2c8687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:09.459295 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-87a5e209-a5fd-4de2-86a4-82633b2c8687 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:09.460494 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-87a5e209-a5fd-4de2-86a4-82633b2c8687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:09.473806 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de9888-d5eb-469b-afd3-0563ab4820da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:09.474479 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de9888-d5eb-469b-afd3-0563ab4820da None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:09.474756 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de9888-d5eb-469b-afd3-0563ab4820da None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:09.475025 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de9888-d5eb-469b-afd3-0563ab4820da None admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:09.475745 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26de9888-d5eb-469b-afd3-0563ab4820da None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:09.476044 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26de9888-d5eb-469b-afd3-0563ab4820da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:09.477546 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26de9888-d5eb-469b-afd3-0563ab4820da None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:09.488343 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:09.495690 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 276/571] 10.222.0.52 () {56 vars in 1137 bytes} [Mon Aug 29 22:29:09 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:09.496495 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de9888-d5eb-469b-afd3-0563ab4820da None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:09.537607 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 296/572] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:29:09.538457 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a5e209-a5fd-4de2-86a4-82633b2c8687 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:09.599961 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:09.612210 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de9888-d5eb-469b-afd3-0563ab4820da None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:09.648575 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fbffc7-e175-4ebf-a045-6ed9e41f11a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:09.649212 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fbffc7-e175-4ebf-a045-6ed9e41f11a1 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:09.649400 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a5e209-a5fd-4de2-86a4-82633b2c8687 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:09.649502 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fbffc7-e175-4ebf-a045-6ed9e41f11a1 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:09.649751 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fbffc7-e175-4ebf-a045-6ed9e41f11a1 None admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:09.650392 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fbffc7-e175-4ebf-a045-6ed9e41f11a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:09.650690 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fbffc7-e175-4ebf-a045-6ed9e41f11a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:09.652079 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fbffc7-e175-4ebf-a045-6ed9e41f11a1 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:09.663632 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 277/573] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:09.668335 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fbffc7-e175-4ebf-a045-6ed9e41f11a1 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:09.691817 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1cf705-945f-45f3-b36c-9c50f0f7d7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:09.692682 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1cf705-945f-45f3-b36c-9c50f0f7d7dd None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:09.693241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1cf705-945f-45f3-b36c-9c50f0f7d7dd None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:09.693653 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1cf705-945f-45f3-b36c-9c50f0f7d7dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:09.704523 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1cf705-945f-45f3-b36c-9c50f0f7d7dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:09.706486 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1cf705-945f-45f3-b36c-9c50f0f7d7dd None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:09.722257 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-0f1cf705-945f-45f3-b36c-9c50f0f7d7dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:09.722257 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:09.722257 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:09.722257 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:09.722257 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:09.722257 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:09.722257 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:09.722257 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:09.722257 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:09.722257 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:09.722257 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:09.722257 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:09.722257 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:09.722257 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:09.722257 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:09.722257 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:09.723361 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:09.723361 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:09.723361 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:09.723361 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:09.723361 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:09.723361 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:09.723361 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:09.723361 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:09.723361 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:09.723361 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:09.723361 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:09.723361 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:09.723361 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:09.723361 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:09.723361 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:09.723361 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:09.723361 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:09.724434 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:09.724434 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:09.724434 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:09.726153 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 297/574] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:09.727269 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1cf705-945f-45f3-b36c-9c50f0f7d7dd None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:09.747213 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:09.773217 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fbffc7-e175-4ebf-a045-6ed9e41f11a1 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:09.810748 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc7a2f-4ac0-4b5b-865e-c39ff57f9f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:09.811357 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc7a2f-4ac0-4b5b-865e-c39ff57f9f82 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:09.811637 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc7a2f-4ac0-4b5b-865e-c39ff57f9f82 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:09.811910 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc7a2f-4ac0-4b5b-865e-c39ff57f9f82 None admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:09.812607 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdc7a2f-4ac0-4b5b-865e-c39ff57f9f82 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:09.812937 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdc7a2f-4ac0-4b5b-865e-c39ff57f9f82 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:09.814370 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdc7a2f-4ac0-4b5b-865e-c39ff57f9f82 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:09.827023 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 278/575] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Aug 29 22:29:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:09.827815 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc7a2f-4ac0-4b5b-865e-c39ff57f9f82 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:09.836660 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1cf705-945f-45f3-b36c-9c50f0f7d7dd None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:09.838439 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:09.869958 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6bffa5-8039-4b24-b66a-760ea8951282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:09.870582 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6bffa5-8039-4b24-b66a-760ea8951282 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:09.870850 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6bffa5-8039-4b24-b66a-760ea8951282 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:09.871121 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6bffa5-8039-4b24-b66a-760ea8951282 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:09.882061 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6bffa5-8039-4b24-b66a-760ea8951282 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:09.883523 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6bffa5-8039-4b24-b66a-760ea8951282 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:09.897689 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-bc6bffa5-8039-4b24-b66a-760ea8951282 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:09.897689 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:09.897689 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:09.897689 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:09.897689 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:09.897689 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:09.897689 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:09.897689 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:09.897689 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:09.897689 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:09.897689 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:09.897689 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:09.897689 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:09.897689 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:09.897689 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:09.897689 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:09.898843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:09.898843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:09.898843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:09.898843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:09.898843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:09.898843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:09.898843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:09.898843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:09.898843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:09.898843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:09.898843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:09.898843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:09.898843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:09.898843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:09.898843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:09.898843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:09.898843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:09.899911 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:09.899911 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:09.899911 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:09.899911 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 298/576] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:09.899911 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6bffa5-8039-4b24-b66a-760ea8951282 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:09.932697 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc7a2f-4ac0-4b5b-865e-c39ff57f9f82 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:09.964433 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:09.966966 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce6146c-c5a6-4456-94ea-d7bd7e2788ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:09.967582 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce6146c-c5a6-4456-94ea-d7bd7e2788ca None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:09.967864 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce6146c-c5a6-4456-94ea-d7bd7e2788ca None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:09.968137 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce6146c-c5a6-4456-94ea-d7bd7e2788ca None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:09.975321 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce6146c-c5a6-4456-94ea-d7bd7e2788ca None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:09.976840 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce6146c-c5a6-4456-94ea-d7bd7e2788ca None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:09.990270 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-5ce6146c-c5a6-4456-94ea-d7bd7e2788ca None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 22:29:09.990270 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:09.990270 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:09.990270 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:09.990270 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:09.990270 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:09.990270 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:09.990270 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:09.990270 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:09.990270 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:09.990270 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:09.990270 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:09.990270 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:29:09.990270 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:29:09.990270 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:29:09.990270 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:29:09.991392 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:09.991392 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:09.991392 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:29:09.991392 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:09.991392 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:29:09.991392 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:09.991392 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:09.991392 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:09.991392 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:09.991392 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:09.991392 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:09.991392 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:09.991392 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:09.991392 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:09.991392 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:09.991392 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:09.991392 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:09.991392 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:09.992352 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:09.992352 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:09.992352 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:29:09.992352 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:29:09.992352 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:29:09.992352 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:29:09.992352 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:29:09.992352 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:29:09.992352 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 22:29:09.992352 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:09.992352 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 279/577] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:29:09 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:09.992352 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce6146c-c5a6-4456-94ea-d7bd7e2788ca None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:10.001308 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6bffa5-8039-4b24-b66a-760ea8951282 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:10.055568 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3651a7-fcf4-4c47-8a94-b1f031861049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:10.056171 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3651a7-fcf4-4c47-8a94-b1f031861049 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:10.056459 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3651a7-fcf4-4c47-8a94-b1f031861049 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:10.056758 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3651a7-fcf4-4c47-8a94-b1f031861049 None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:10.057399 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3651a7-fcf4-4c47-8a94-b1f031861049 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:10.057704 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3651a7-fcf4-4c47-8a94-b1f031861049 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:10.059069 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3651a7-fcf4-4c47-8a94-b1f031861049 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:10.071055 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 299/578] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:10.071856 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3651a7-fcf4-4c47-8a94-b1f031861049 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:10.090731 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:10.124492 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce6146c-c5a6-4456-94ea-d7bd7e2788ca None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:10.157801 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ccfc31-1a06-4db6-a605-4cb98d1ac9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:10.158917 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ccfc31-1a06-4db6-a605-4cb98d1ac9af None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:10.159191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ccfc31-1a06-4db6-a605-4cb98d1ac9af None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:10.159538 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ccfc31-1a06-4db6-a605-4cb98d1ac9af None admin] PATH_INFO: `/v3/roles/service` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:10.169402 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ccfc31-1a06-4db6-a605-4cb98d1ac9af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:29:10.174470 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3651a7-fcf4-4c47-8a94-b1f031861049 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:10.175103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-59ccfc31-1a06-4db6-a605-4cb98d1ac9af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:29:10.175103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:10.175103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:10.175103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:10.175103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:10.175103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:10.175103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:10.175103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:10.175103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:10.175103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:10.175103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:10.175103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:10.175103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:29:10.175103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:29:10.175103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:29:10.175103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:29:10.176182 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:29:10.176182 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:29:10.176182 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:29:10.176182 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:29:10.176182 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:10.176182 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:10.176182 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:10.176182 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:10.176182 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:10.176182 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:10.176182 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:10.176182 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:10.176182 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:10.176182 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:10.176182 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:10.176182 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:10.181977 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:10.181977 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:10.181977 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:10.181977 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:10.181977 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:29:10.181977 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:29:10.181977 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:29:10.181977 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:29:10.181977 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:29:10.181977 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:29:10.181977 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:29:10.181977 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:10.181977 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 280/579] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Aug 29 22:29:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:10.181977 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ccfc31-1a06-4db6-a605-4cb98d1ac9af None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:10.201211 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:10.210211 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86200186-60e4-4cbd-8e6b-60ba99c04d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:10.210818 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86200186-60e4-4cbd-8e6b-60ba99c04d19 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:10.216694 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86200186-60e4-4cbd-8e6b-60ba99c04d19 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:10.216694 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86200186-60e4-4cbd-8e6b-60ba99c04d19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:10.216694 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86200186-60e4-4cbd-8e6b-60ba99c04d19 None admin] RBAC: Adding query filter params (role.id=db83c2f8e21945c3ab9642c620d29a23, scope.project.id=ee84c3340f2641ad963655a7a212ef96, user.id=57c175ac90e545358746176972ff39a3) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:10.216694 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86200186-60e4-4cbd-8e6b-60ba99c04d19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:10.216694 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86200186-60e4-4cbd-8e6b-60ba99c04d19 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:10.219274 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 300/580] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:29:10 2022] GET /identity/v3/role_assignments?user.id=57c175ac90e545358746176972ff39a3&scope.project.id=ee84c3340f2641ad963655a7a212ef96&role.id=db83c2f8e21945c3ab9642c620d29a23 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:10.220033 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86200186-60e4-4cbd-8e6b-60ba99c04d19 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:10.297916 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ccfc31-1a06-4db6-a605-4cb98d1ac9af None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:10.329599 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:10.339848 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95bdf3-fbe0-4e57-8373-a5fb0e20b73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:10.340766 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95bdf3-fbe0-4e57-8373-a5fb0e20b73f None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:10.342658 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95bdf3-fbe0-4e57-8373-a5fb0e20b73f None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:10.342658 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95bdf3-fbe0-4e57-8373-a5fb0e20b73f None admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:10.342658 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc95bdf3-fbe0-4e57-8373-a5fb0e20b73f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:10.342908 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc95bdf3-fbe0-4e57-8373-a5fb0e20b73f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:10.344556 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc95bdf3-fbe0-4e57-8373-a5fb0e20b73f None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:10.348832 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86200186-60e4-4cbd-8e6b-60ba99c04d19 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:10.363477 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 281/581] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:10.364459 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95bdf3-fbe0-4e57-8373-a5fb0e20b73f None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:10.392378 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68f699-2408-4b31-ba41-c2001cf2012a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:10.393757 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68f699-2408-4b31-ba41-c2001cf2012a None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:10.394034 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68f699-2408-4b31-ba41-c2001cf2012a None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:10.394302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68f699-2408-4b31-ba41-c2001cf2012a None admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:10.395504 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee68f699-2408-4b31-ba41-c2001cf2012a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:10.395824 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee68f699-2408-4b31-ba41-c2001cf2012a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:10.397710 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee68f699-2408-4b31-ba41-c2001cf2012a None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:10.425654 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 301/582] 10.222.0.52 () {56 vars in 1137 bytes} [Mon Aug 29 22:29:10 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:10.425654 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68f699-2408-4b31-ba41-c2001cf2012a None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:10.472189 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95bdf3-fbe0-4e57-8373-a5fb0e20b73f None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:10.482706 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:10.512407 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd73ef53-0aa7-43b7-9c10-987b46cbdbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:10.513069 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd73ef53-0aa7-43b7-9c10-987b46cbdbe9 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:10.513370 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd73ef53-0aa7-43b7-9c10-987b46cbdbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:10.513637 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd73ef53-0aa7-43b7-9c10-987b46cbdbe9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:10.521622 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd73ef53-0aa7-43b7-9c10-987b46cbdbe9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:10.523075 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd73ef53-0aa7-43b7-9c10-987b46cbdbe9 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:10.535395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-dd73ef53-0aa7-43b7-9c10-987b46cbdbe9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 22:29:10.535395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:10.535395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:10.535395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:10.535395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:10.535395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:10.535395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:10.535395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:10.535395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:10.535395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:10.535395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:10.535395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:10.535395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:29:10.535395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:29:10.535395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:29:10.535395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:29:10.536358 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:10.536358 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:10.536358 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:29:10.536358 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:10.536358 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:29:10.536358 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:10.536358 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:10.536358 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:10.536358 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:10.536358 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:10.536358 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:10.536358 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:10.536358 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:10.536358 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:10.536358 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:10.536358 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:10.536358 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:10.536358 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:10.537339 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:10.537339 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:10.537339 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:29:10.537339 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:29:10.537339 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:29:10.537339 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:29:10.537339 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:29:10.537339 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:29:10.537339 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 22:29:10.537339 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:10.537339 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 282/583] 10.222.0.52 () {56 vars in 1093 bytes} [Mon Aug 29 22:29:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:10.537339 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd73ef53-0aa7-43b7-9c10-987b46cbdbe9 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:10.548375 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68f699-2408-4b31-ba41-c2001cf2012a None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:10.586408 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5637a284-3faf-4d0b-a033-525f02872624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:10.587379 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637a284-3faf-4d0b-a033-525f02872624 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:10.587672 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637a284-3faf-4d0b-a033-525f02872624 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:10.587944 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637a284-3faf-4d0b-a033-525f02872624 None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:10.588957 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5637a284-3faf-4d0b-a033-525f02872624 None admin] RBAC: Adding query filter params (name=service) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:10.589265 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5637a284-3faf-4d0b-a033-525f02872624 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:10.590784 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5637a284-3faf-4d0b-a033-525f02872624 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:10.595041 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:10.604540 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 302/584] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Aug 29 22:29:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:10.604540 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5637a284-3faf-4d0b-a033-525f02872624 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:10.674969 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd73ef53-0aa7-43b7-9c10-987b46cbdbe9 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:10.714911 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3fa198-7c16-4c7b-bf87-e4a38fd9724f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:10.715548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3fa198-7c16-4c7b-bf87-e4a38fd9724f None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:10.715828 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3fa198-7c16-4c7b-bf87-e4a38fd9724f None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:10.716102 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3fa198-7c16-4c7b-bf87-e4a38fd9724f None admin] PATH_INFO: `/v3/projects/service` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:10.724660 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3fa198-7c16-4c7b-bf87-e4a38fd9724f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:10.726169 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3fa198-7c16-4c7b-bf87-e4a38fd9724f None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:10.732935 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5637a284-3faf-4d0b-a033-525f02872624 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:10.739625 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-2b3fa198-7c16-4c7b-bf87-e4a38fd9724f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:10.739625 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:10.739625 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:10.739625 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:10.739625 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:10.739625 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:10.739625 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:10.739625 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:10.739625 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:10.739625 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:10.739625 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:10.739625 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:10.739625 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:29:10.739625 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:29:10.739625 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:29:10.740531 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:29:10.740531 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:10.740531 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:10.740531 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:10.740531 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:10.740531 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:10.740531 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:10.740531 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:10.740531 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:10.740531 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:10.740531 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:10.740531 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:10.740531 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:10.740531 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:10.740531 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:10.740531 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:10.740531 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:10.741514 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:29:10.741514 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:29:10.741514 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:29:10.741514 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:29:10.741514 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:29:10.741514 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:29:10.741514 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:10.741514 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:10.741514 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 283/585] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:29:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:10.741999 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3fa198-7c16-4c7b-bf87-e4a38fd9724f None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:10.749355 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:10.776591 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31d4fba-b15f-4a58-99be-81e290a1b4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:10.777228 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d4fba-b15f-4a58-99be-81e290a1b4d7 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:10.777509 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d4fba-b15f-4a58-99be-81e290a1b4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:10.777785 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d4fba-b15f-4a58-99be-81e290a1b4d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:10.790425 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31d4fba-b15f-4a58-99be-81e290a1b4d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:10.792186 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31d4fba-b15f-4a58-99be-81e290a1b4d7 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:10.810118 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-b31d4fba-b15f-4a58-99be-81e290a1b4d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:10.810118 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:10.810118 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:10.810118 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:10.810118 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:10.810118 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:10.810118 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:10.810118 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:10.810118 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:10.810118 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:10.810118 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:10.810118 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:10.810118 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:10.810118 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:10.810118 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:10.810118 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:10.811086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:10.811086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:10.811086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:10.811086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:10.811086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:10.811086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:10.811086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:10.811086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:10.811086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:10.811086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:10.811086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:10.811086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:10.811086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:10.811086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:10.811086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:10.811086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:10.811086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:10.811985 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:10.811985 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:10.811985 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:10.811985 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 303/586] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:10.812836 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31d4fba-b15f-4a58-99be-81e290a1b4d7 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:10.857654 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3fa198-7c16-4c7b-bf87-e4a38fd9724f None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:10.868396 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:10.894385 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371a0584-6259-420b-8fcb-06967e1f4512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:10.894991 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a0584-6259-420b-8fcb-06967e1f4512 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:10.895271 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a0584-6259-420b-8fcb-06967e1f4512 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:10.895541 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a0584-6259-420b-8fcb-06967e1f4512 None admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:10.896273 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371a0584-6259-420b-8fcb-06967e1f4512 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:10.896698 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371a0584-6259-420b-8fcb-06967e1f4512 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:10.898292 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371a0584-6259-420b-8fcb-06967e1f4512 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:10.922424 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 284/587] 10.222.0.52 () {56 vars in 1133 bytes} [Mon Aug 29 22:29:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:10.922424 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371a0584-6259-420b-8fcb-06967e1f4512 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:10.926582 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31d4fba-b15f-4a58-99be-81e290a1b4d7 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:10.967449 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda9ff6-a4e0-4229-8de8-e5e1ae85ce5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:10.968984 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda9ff6-a4e0-4229-8de8-e5e1ae85ce5d None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:10.969479 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda9ff6-a4e0-4229-8de8-e5e1ae85ce5d None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:10.969973 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda9ff6-a4e0-4229-8de8-e5e1ae85ce5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:10.988371 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abda9ff6-a4e0-4229-8de8-e5e1ae85ce5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:10.989865 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abda9ff6-a4e0-4229-8de8-e5e1ae85ce5d None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:11.004319 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-abda9ff6-a4e0-4229-8de8-e5e1ae85ce5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:11.004319 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:11.004319 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:11.004319 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:11.004319 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:11.004319 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:11.004319 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:11.004319 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:11.004319 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:11.004319 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:11.004319 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:11.004319 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:11.004319 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:11.004319 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:11.004319 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:11.004319 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:11.005471 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:11.005471 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:11.005471 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:11.005471 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:11.005471 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:11.005471 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:11.005471 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:11.005471 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:11.005471 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:11.005471 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:11.005471 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:11.005471 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:11.005471 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:11.005471 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:11.005471 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:11.005471 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:11.005471 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:11.006363 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:11.006363 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:11.006363 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:11.006363 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 304/588] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:11.006363 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda9ff6-a4e0-4229-8de8-e5e1ae85ce5d None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:11.027109 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:11.051834 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371a0584-6259-420b-8fcb-06967e1f4512 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:11.094862 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6f9639-fd3d-47b4-b8c2-4b57702ebb32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:11.095370 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f9639-fd3d-47b4-b8c2-4b57702ebb32 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:11.096209 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f9639-fd3d-47b4-b8c2-4b57702ebb32 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:11.096585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f9639-fd3d-47b4-b8c2-4b57702ebb32 None admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:11.097315 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6f9639-fd3d-47b4-b8c2-4b57702ebb32 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:11.097619 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6f9639-fd3d-47b4-b8c2-4b57702ebb32 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:11.099213 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6f9639-fd3d-47b4-b8c2-4b57702ebb32 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:11.112401 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 285/589] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Aug 29 22:29:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:11.113281 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6f9639-fd3d-47b4-b8c2-4b57702ebb32 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:11.132955 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda9ff6-a4e0-4229-8de8-e5e1ae85ce5d None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:11.143632 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:11.170146 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9117fa-efea-4a2e-9fb1-0e3e2b7541e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:11.170756 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9117fa-efea-4a2e-9fb1-0e3e2b7541e3 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:11.171036 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9117fa-efea-4a2e-9fb1-0e3e2b7541e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:11.171308 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9117fa-efea-4a2e-9fb1-0e3e2b7541e3 None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:11.172103 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9117fa-efea-4a2e-9fb1-0e3e2b7541e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:11.172405 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9117fa-efea-4a2e-9fb1-0e3e2b7541e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:11.173840 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9117fa-efea-4a2e-9fb1-0e3e2b7541e3 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:11.186427 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 305/590] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:11.187506 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9117fa-efea-4a2e-9fb1-0e3e2b7541e3 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:11.227716 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6f9639-fd3d-47b4-b8c2-4b57702ebb32 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:11.261874 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c2711b-504c-4a36-bf86-d89a907a4a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:11.262500 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2711b-504c-4a36-bf86-d89a907a4a10 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:11.262781 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2711b-504c-4a36-bf86-d89a907a4a10 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:11.263054 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2711b-504c-4a36-bf86-d89a907a4a10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:11.270660 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:11.275606 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c2711b-504c-4a36-bf86-d89a907a4a10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:11.277091 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c2711b-504c-4a36-bf86-d89a907a4a10 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:11.292347 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-e3c2711b-504c-4a36-bf86-d89a907a4a10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:11.292347 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:11.292347 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:11.292347 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:11.292347 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:11.292347 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:11.292347 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:11.292347 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:11.292347 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:11.292347 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:11.292347 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:11.292347 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:11.292347 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:11.292347 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:11.292347 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:11.292347 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:11.293665 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:11.293665 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:11.293665 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:11.293665 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:11.293665 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:11.293665 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:11.293665 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:11.293665 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:11.293665 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:11.293665 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:11.293665 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:11.293665 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:11.293665 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:11.293665 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:11.293665 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:11.293665 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:11.293665 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:11.294554 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:11.294554 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:11.294554 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:11.294554 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 286/591] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:11.294778 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c2711b-504c-4a36-bf86-d89a907a4a10 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:11.306901 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9117fa-efea-4a2e-9fb1-0e3e2b7541e3 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:11.344831 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e4eef8-157c-45a5-b269-fb030f0d7562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:11.345476 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e4eef8-157c-45a5-b269-fb030f0d7562 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:11.345807 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e4eef8-157c-45a5-b269-fb030f0d7562 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:11.346146 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e4eef8-157c-45a5-b269-fb030f0d7562 None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:11.346978 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e4eef8-157c-45a5-b269-fb030f0d7562 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:11.347285 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e4eef8-157c-45a5-b269-fb030f0d7562 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:11.348936 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e4eef8-157c-45a5-b269-fb030f0d7562 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:11.367068 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 306/592] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:11.367068 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e4eef8-157c-45a5-b269-fb030f0d7562 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:11.381682 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:11.408254 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c2711b-504c-4a36-bf86-d89a907a4a10 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:11.447946 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef4e08-34ca-467b-8f5b-12ef45bf4001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:11.448600 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef4e08-34ca-467b-8f5b-12ef45bf4001 None admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:11.448922 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef4e08-34ca-467b-8f5b-12ef45bf4001 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:11.449556 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef4e08-34ca-467b-8f5b-12ef45bf4001 None admin] PATH_INFO: `/v3/projects/ee84c3340f2641ad963655a7a212ef96/users/9c2569e7cd774199aa5750fd6a850060/roles/db83c2f8e21945c3ab9642c620d29a23` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:11.465483 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef4e08-34ca-467b-8f5b-12ef45bf4001 None admin] RBAC: Authorizing `identity:create_grant(project_id=ee84c3340f2641ad963655a7a212ef96, user_id=9c2569e7cd774199aa5750fd6a850060, role_id=db83c2f8e21945c3ab9642c620d29a23)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:11.466911 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef4e08-34ca-467b-8f5b-12ef45bf4001 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:11.473144 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e4eef8-157c-45a5-b269-fb030f0d7562 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:11.507437 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14367c9-5e93-416d-be5b-bf236978a7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:11.508059 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14367c9-5e93-416d-be5b-bf236978a7d8 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:11.508333 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14367c9-5e93-416d-be5b-bf236978a7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:11.508602 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14367c9-5e93-416d-be5b-bf236978a7d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:11.510569 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:11.515835 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14367c9-5e93-416d-be5b-bf236978a7d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:11.517323 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14367c9-5e93-416d-be5b-bf236978a7d8 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:11.528472 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-e14367c9-5e93-416d-be5b-bf236978a7d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:11.528472 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:11.528472 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:11.528472 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:11.528472 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:11.528472 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:11.528472 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:11.528472 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:11.528472 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:11.528472 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:11.528472 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:11.528472 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:11.528472 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:29:11.528472 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:29:11.528472 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:29:11.529550 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:29:11.529550 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:11.529550 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:11.529550 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:11.529550 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:11.529550 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:11.529550 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:11.529550 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:11.529550 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:11.529550 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:11.529550 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:11.529550 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:11.529550 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:11.529550 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:11.529550 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:11.529550 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:11.529550 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:11.530655 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:29:11.530655 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:29:11.530655 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:29:11.530655 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:29:11.530655 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:29:11.530655 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:29:11.530655 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:11.530655 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:11.535745 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 307/593] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:29:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:11.535745 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14367c9-5e93-416d-be5b-bf236978a7d8 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:11.557021 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 287/594] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Aug 29 22:29:11 2022] PUT /identity/v3/projects/ee84c3340f2641ad963655a7a212ef96/users/9c2569e7cd774199aa5750fd6a850060/roles/db83c2f8e21945c3ab9642c620d29a23 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:29:11.558058 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef4e08-34ca-467b-8f5b-12ef45bf4001 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:11.662969 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14367c9-5e93-416d-be5b-bf236978a7d8 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:11.713744 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef4e08-34ca-467b-8f5b-12ef45bf4001 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:11.713939 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e4ab9-17b7-45f8-9ce0-8b6b3981c318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:11.713939 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e4ab9-17b7-45f8-9ce0-8b6b3981c318 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:11.713939 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e4ab9-17b7-45f8-9ce0-8b6b3981c318 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:11.713939 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e4ab9-17b7-45f8-9ce0-8b6b3981c318 None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:11.714255 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7e4ab9-17b7-45f8-9ce0-8b6b3981c318 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:11.714255 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7e4ab9-17b7-45f8-9ce0-8b6b3981c318 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:11.714255 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7e4ab9-17b7-45f8-9ce0-8b6b3981c318 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:11.727047 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 308/595] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:11.727892 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e4ab9-17b7-45f8-9ce0-8b6b3981c318 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:11.761019 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:11.771535 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c245de79-5ab0-4d08-b1c7-3f1ebe9b5b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:11.772331 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c245de79-5ab0-4d08-b1c7-3f1ebe9b5b94 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:11.772615 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c245de79-5ab0-4d08-b1c7-3f1ebe9b5b94 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:11.773732 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c245de79-5ab0-4d08-b1c7-3f1ebe9b5b94 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:11.782129 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c245de79-5ab0-4d08-b1c7-3f1ebe9b5b94 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:11.785311 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c245de79-5ab0-4d08-b1c7-3f1ebe9b5b94 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:11.802780 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:11.803535 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-c245de79-5ab0-4d08-b1c7-3f1ebe9b5b94 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 22:29:11.803535 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:11.803535 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:11.803535 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:11.803535 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:11.803535 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:11.803535 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:11.803535 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:11.803535 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:11.803535 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:11.803535 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:11.803535 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:11.803535 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:29:11.803535 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:29:11.803535 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:29:11.803535 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:29:11.804534 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:11.804534 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:11.804534 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:29:11.804534 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:11.804534 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:29:11.804534 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:11.804534 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:11.804534 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:11.804534 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:11.804534 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:11.804534 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:11.804534 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:11.804534 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:11.804534 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:11.804534 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:11.804534 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:11.804534 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:11.804534 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:11.805523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:11.805523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:11.805523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:29:11.805523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:29:11.805523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:29:11.805523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:29:11.805523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:29:11.805523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:29:11.805523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 22:29:11.805523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:11.808433 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 288/596] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Aug 29 22:29:11 2022] GET /identity/v3/users/neutron => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:11.809720 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c245de79-5ab0-4d08-b1c7-3f1ebe9b5b94 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:11.866120 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e4ab9-17b7-45f8-9ce0-8b6b3981c318 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:11.907722 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd07b29-cc29-487d-b682-a837c285f4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:11.908962 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd07b29-cc29-487d-b682-a837c285f4c5 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:11.909479 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd07b29-cc29-487d-b682-a837c285f4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:11.909843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd07b29-cc29-487d-b682-a837c285f4c5 None admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:11.910548 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd07b29-cc29-487d-b682-a837c285f4c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:11.910861 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd07b29-cc29-487d-b682-a837c285f4c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:11.912307 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd07b29-cc29-487d-b682-a837c285f4c5 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:11.925701 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 309/597] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Aug 29 22:29:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:11.926554 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd07b29-cc29-487d-b682-a837c285f4c5 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:11.942559 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c245de79-5ab0-4d08-b1c7-3f1ebe9b5b94 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:11.989126 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3803f122-5d3f-4339-abdd-6a5ab3163379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:11.990041 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803f122-5d3f-4339-abdd-6a5ab3163379 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:11.990382 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803f122-5d3f-4339-abdd-6a5ab3163379 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:11.991091 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803f122-5d3f-4339-abdd-6a5ab3163379 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:12.000202 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3803f122-5d3f-4339-abdd-6a5ab3163379 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:12.004535 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3803f122-5d3f-4339-abdd-6a5ab3163379 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:12.020104 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-3803f122-5d3f-4339-abdd-6a5ab3163379 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:12.020104 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:12.020104 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:12.020104 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:12.020104 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:12.020104 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:12.020104 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:12.020104 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:12.020104 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:12.020104 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:12.020104 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:12.020104 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:12.020104 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:29:12.020104 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:29:12.020104 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:29:12.021354 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:29:12.021354 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:12.021354 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:12.021354 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:12.021354 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:12.021354 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:12.021354 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:12.021354 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:12.021354 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:12.021354 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:12.021354 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:12.021354 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:12.021354 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:12.021354 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:12.021354 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:12.021354 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:12.021354 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:12.022335 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:29:12.022335 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:29:12.022335 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:29:12.022335 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:29:12.022335 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:29:12.022335 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:29:12.022335 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:12.022335 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:12.025649 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 289/598] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:29:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:12.026283 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3803f122-5d3f-4339-abdd-6a5ab3163379 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:12.044975 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd07b29-cc29-487d-b682-a837c285f4c5 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:12.060953 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:12.082915 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc161a4-e74a-45c8-9519-c1ee03c90e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:12.083531 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc161a4-e74a-45c8-9519-c1ee03c90e62 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:12.083816 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc161a4-e74a-45c8-9519-c1ee03c90e62 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:12.084093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc161a4-e74a-45c8-9519-c1ee03c90e62 None admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:12.084853 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc161a4-e74a-45c8-9519-c1ee03c90e62 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:12.085343 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc161a4-e74a-45c8-9519-c1ee03c90e62 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:12.086785 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc161a4-e74a-45c8-9519-c1ee03c90e62 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:12.100823 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:12.111172 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 310/599] 10.222.0.52 () {56 vars in 1139 bytes} [Mon Aug 29 22:29:11 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:12.112687 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc161a4-e74a-45c8-9519-c1ee03c90e62 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:12.169837 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3803f122-5d3f-4339-abdd-6a5ab3163379 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:12.206774 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba76fa0b-f24d-4bea-95af-1b4b6f9938da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:12.207456 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76fa0b-f24d-4bea-95af-1b4b6f9938da None admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:12.207700 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76fa0b-f24d-4bea-95af-1b4b6f9938da None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:12.207968 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76fa0b-f24d-4bea-95af-1b4b6f9938da None admin] PATH_INFO: `/v3/projects/ee84c3340f2641ad963655a7a212ef96/users/6e782c8acaaa42c4bd78ddc7c8cc654b/roles/db83c2f8e21945c3ab9642c620d29a23` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:12.223182 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba76fa0b-f24d-4bea-95af-1b4b6f9938da None admin] RBAC: Authorizing `identity:create_grant(project_id=ee84c3340f2641ad963655a7a212ef96, user_id=6e782c8acaaa42c4bd78ddc7c8cc654b, role_id=db83c2f8e21945c3ab9642c620d29a23)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:12.224506 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc161a4-e74a-45c8-9519-c1ee03c90e62 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:12.224642 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba76fa0b-f24d-4bea-95af-1b4b6f9938da None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:12.261032 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f839d6-7057-4c5c-9cfe-d098dd554c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:12.261634 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f839d6-7057-4c5c-9cfe-d098dd554c11 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:12.261906 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f839d6-7057-4c5c-9cfe-d098dd554c11 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:12.262173 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f839d6-7057-4c5c-9cfe-d098dd554c11 None admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:12.262883 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f839d6-7057-4c5c-9cfe-d098dd554c11 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:12.263183 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f839d6-7057-4c5c-9cfe-d098dd554c11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:12.264591 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f839d6-7057-4c5c-9cfe-d098dd554c11 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:12.276581 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 311/600] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Aug 29 22:29:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:12.277431 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f839d6-7057-4c5c-9cfe-d098dd554c11 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:12.309620 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:12.334913 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 290/601] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Aug 29 22:29:12 2022] PUT /identity/v3/projects/ee84c3340f2641ad963655a7a212ef96/users/6e782c8acaaa42c4bd78ddc7c8cc654b/roles/db83c2f8e21945c3ab9642c620d29a23 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:29:12.334913 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f239d-bbf6-4cb5-ac39-2062e364ff6a None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:12.334913 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f239d-bbf6-4cb5-ac39-2062e364ff6a None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:12.334913 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f239d-bbf6-4cb5-ac39-2062e364ff6a None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:12.334913 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 291/602] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:29:12.334913 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f239d-bbf6-4cb5-ac39-2062e364ff6a None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:12.404594 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f839d6-7057-4c5c-9cfe-d098dd554c11 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:12.444839 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad5b05d-ac3c-4c1d-ba49-1d0d88611117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:12.445578 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5b05d-ac3c-4c1d-ba49-1d0d88611117 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:12.445866 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5b05d-ac3c-4c1d-ba49-1d0d88611117 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:12.446142 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5b05d-ac3c-4c1d-ba49-1d0d88611117 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:12.460323 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad5b05d-ac3c-4c1d-ba49-1d0d88611117 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:12.465445 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:12.468335 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad5b05d-ac3c-4c1d-ba49-1d0d88611117 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:12.471159 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f239d-bbf6-4cb5-ac39-2062e364ff6a None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:12.491283 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-0ad5b05d-ac3c-4c1d-ba49-1d0d88611117 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:12.491283 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:12.491283 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:12.491283 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:12.491283 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:12.491283 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:12.491283 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:12.491283 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:12.491283 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:12.491283 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:12.491283 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:12.491283 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:12.491283 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:12.491283 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:12.491283 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:12.491283 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:12.492559 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:12.492559 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:12.492559 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:12.492559 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:12.492559 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:12.492559 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:12.492559 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:12.492559 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:12.492559 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:12.492559 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:12.492559 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:12.492559 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:12.492559 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:12.492559 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:12.492559 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:12.492559 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:12.492559 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:12.493682 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:12.493682 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:12.493682 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:12.497434 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 312/603] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:12.499179 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c282f6-5030-416a-9a9b-aaa29b62f3f8 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:12.499461 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c282f6-5030-416a-9a9b-aaa29b62f3f8 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:12.499733 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c282f6-5030-416a-9a9b-aaa29b62f3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:12.517921 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82ee530-c27a-40ff-85ef-2aa0eefb5ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:12.518589 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82ee530-c27a-40ff-85ef-2aa0eefb5ce3 None admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:12.518869 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82ee530-c27a-40ff-85ef-2aa0eefb5ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:12.519146 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82ee530-c27a-40ff-85ef-2aa0eefb5ce3 None admin] PATH_INFO: `/v3/projects/ee84c3340f2641ad963655a7a212ef96/users/bdbf961e02934e88af3a72d6218f32fa/roles/db83c2f8e21945c3ab9642c620d29a23` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:12.527697 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-34c282f6-5030-416a-9a9b-aaa29b62f3f8 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:12.528925 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-34c282f6-5030-416a-9a9b-aaa29b62f3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:12.546785 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82ee530-c27a-40ff-85ef-2aa0eefb5ce3 None admin] RBAC: Authorizing `identity:create_grant(project_id=ee84c3340f2641ad963655a7a212ef96, user_id=bdbf961e02934e88af3a72d6218f32fa, role_id=db83c2f8e21945c3ab9642c620d29a23)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:12.548248 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82ee530-c27a-40ff-85ef-2aa0eefb5ce3 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:12.613116 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 313/604] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:29:12.614087 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c282f6-5030-416a-9a9b-aaa29b62f3f8 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:12.632380 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 292/605] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Aug 29 22:29:12 2022] PUT /identity/v3/projects/ee84c3340f2641ad963655a7a212ef96/users/bdbf961e02934e88af3a72d6218f32fa/roles/db83c2f8e21945c3ab9642c620d29a23 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:29:12.638231 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:12.775217 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c282f6-5030-416a-9a9b-aaa29b62f3f8 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:12.824258 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b38c4-a112-4932-bb7b-2de80edaab4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:12.824917 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b38c4-a112-4932-bb7b-2de80edaab4a None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:12.825201 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b38c4-a112-4932-bb7b-2de80edaab4a None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:12.825470 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b38c4-a112-4932-bb7b-2de80edaab4a None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:12.826133 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378b38c4-a112-4932-bb7b-2de80edaab4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:12.826429 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378b38c4-a112-4932-bb7b-2de80edaab4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:12.827827 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378b38c4-a112-4932-bb7b-2de80edaab4a None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:12.841378 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 314/606] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:12.846855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82ee530-c27a-40ff-85ef-2aa0eefb5ce3 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:12.962511 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82ee530-c27a-40ff-85ef-2aa0eefb5ce3 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:12.995811 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b38c4-a112-4932-bb7b-2de80edaab4a None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:13.012310 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09797fe-7ebe-4d65-b0d1-7ba1406b716f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:13.013400 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09797fe-7ebe-4d65-b0d1-7ba1406b716f None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:13.013914 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09797fe-7ebe-4d65-b0d1-7ba1406b716f None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:13.014418 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09797fe-7ebe-4d65-b0d1-7ba1406b716f None admin] PATH_INFO: `/v3/projects/service` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:13.034645 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09797fe-7ebe-4d65-b0d1-7ba1406b716f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:13.036491 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09797fe-7ebe-4d65-b0d1-7ba1406b716f None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:13.059547 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-e09797fe-7ebe-4d65-b0d1-7ba1406b716f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:13.059547 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:13.059547 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:13.059547 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:13.059547 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:13.059547 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:13.059547 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:13.059547 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:13.059547 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:13.059547 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:13.059547 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:13.059547 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:13.059547 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:29:13.059547 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:29:13.059547 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:29:13.060524 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:29:13.060524 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:13.060524 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:13.060524 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:13.060524 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:13.060524 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:13.060524 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:13.060524 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:13.060524 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:13.060524 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:13.060524 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:13.060524 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:13.060524 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:13.060524 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:13.060524 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:13.060524 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:13.060524 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:13.061448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:29:13.061448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:29:13.061448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:29:13.061448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:29:13.061448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:29:13.061448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:29:13.061448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:13.061448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:13.069683 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 293/607] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:29:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:13.075431 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09797fe-7ebe-4d65-b0d1-7ba1406b716f None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:13.091604 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:13.093212 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:13.127678 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b38c4-a112-4932-bb7b-2de80edaab4a None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:13.176781 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42ec3e-7310-4551-a6c3-203545c082f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:13.177746 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42ec3e-7310-4551-a6c3-203545c082f8 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:13.178214 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42ec3e-7310-4551-a6c3-203545c082f8 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:13.178964 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42ec3e-7310-4551-a6c3-203545c082f8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:13.191627 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b42ec3e-7310-4551-a6c3-203545c082f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:29:13.197962 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-3b42ec3e-7310-4551-a6c3-203545c082f8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:29:13.197962 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:13.197962 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:13.197962 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:13.197962 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:13.197962 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:13.197962 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:13.197962 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:13.197962 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:13.197962 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:13.197962 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:13.197962 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:13.197962 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:29:13.197962 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:29:13.197962 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:29:13.197962 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:29:13.199152 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:29:13.199152 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:29:13.199152 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:29:13.199152 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:29:13.199152 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:13.199152 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:13.199152 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:13.199152 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:13.199152 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:13.199152 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:13.199152 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:13.199152 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:13.199152 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:13.199152 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:13.199152 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:13.199152 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:13.200241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:13.200241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:13.200241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:13.200241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:13.200241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:29:13.200241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:29:13.200241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:29:13.200241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:29:13.200241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:29:13.200241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:29:13.200241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:29:13.200241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:13.203769 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 315/608] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Aug 29 22:29:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:13.211456 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42ec3e-7310-4551-a6c3-203545c082f8 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:13.357641 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42ec3e-7310-4551-a6c3-203545c082f8 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:13.360829 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09797fe-7ebe-4d65-b0d1-7ba1406b716f None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:13.401259 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf7c08-245d-44b8-8496-351ec671e7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:13.402163 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf7c08-245d-44b8-8496-351ec671e7c5 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:13.402894 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf7c08-245d-44b8-8496-351ec671e7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:13.403774 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf7c08-245d-44b8-8496-351ec671e7c5 None admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:13.404613 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf7c08-245d-44b8-8496-351ec671e7c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:13.405079 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf7c08-245d-44b8-8496-351ec671e7c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:13.406654 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf7c08-245d-44b8-8496-351ec671e7c5 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:13.421506 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 294/609] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Aug 29 22:29:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:29:13.429650 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf7c08-245d-44b8-8496-351ec671e7c5 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:13.458101 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58583258-d741-49c2-9cbe-e1472f5a0311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:13.458718 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58583258-d741-49c2-9cbe-e1472f5a0311 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:13.458996 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58583258-d741-49c2-9cbe-e1472f5a0311 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:13.459261 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58583258-d741-49c2-9cbe-e1472f5a0311 None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:13.459931 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58583258-d741-49c2-9cbe-e1472f5a0311 None admin] RBAC: Adding query filter params (name=service) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:13.460231 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58583258-d741-49c2-9cbe-e1472f5a0311 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:13.464741 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58583258-d741-49c2-9cbe-e1472f5a0311 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:13.481115 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:13.489875 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 316/610] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Aug 29 22:29:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:13.497702 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58583258-d741-49c2-9cbe-e1472f5a0311 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:13.582300 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:13.649041 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58583258-d741-49c2-9cbe-e1472f5a0311 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:13.652532 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf7c08-245d-44b8-8496-351ec671e7c5 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:13.688817 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d70d240-3d75-4256-93f9-5d56b22c51b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:13.690136 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70d240-3d75-4256-93f9-5d56b22c51b9 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:13.690802 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70d240-3d75-4256-93f9-5d56b22c51b9 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:13.691378 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70d240-3d75-4256-93f9-5d56b22c51b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:13.703778 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d70d240-3d75-4256-93f9-5d56b22c51b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:13.707190 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d70d240-3d75-4256-93f9-5d56b22c51b9 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:13.726980 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e45382d-1027-4bd7-9ebc-9b1c3246aa9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:13.728009 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45382d-1027-4bd7-9ebc-9b1c3246aa9d None admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:13.728489 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45382d-1027-4bd7-9ebc-9b1c3246aa9d None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:13.728988 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45382d-1027-4bd7-9ebc-9b1c3246aa9d None admin] PATH_INFO: `/v3/projects/ee84c3340f2641ad963655a7a212ef96/users/0fe5e8c4cb324ffb9d5b80f18d123727/roles/db83c2f8e21945c3ab9642c620d29a23` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:13.741983 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-3d70d240-3d75-4256-93f9-5d56b22c51b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:13.741983 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:13.741983 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:13.741983 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:13.741983 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:13.741983 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:13.741983 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:13.741983 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:13.741983 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:13.741983 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:13.741983 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:13.741983 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:13.741983 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:13.741983 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:13.741983 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:13.741983 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:13.742985 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:13.742985 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:13.742985 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:13.742985 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:13.742985 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:13.742985 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:13.742985 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:13.742985 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:13.742985 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:13.742985 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:13.742985 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:13.742985 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:13.742985 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:13.742985 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:13.742985 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:13.742985 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:13.742985 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:13.743904 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:13.743904 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:13.743904 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:13.743904 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 317/611] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:29:13.748401 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e45382d-1027-4bd7-9ebc-9b1c3246aa9d None admin] RBAC: Authorizing `identity:create_grant(project_id=ee84c3340f2641ad963655a7a212ef96, user_id=0fe5e8c4cb324ffb9d5b80f18d123727, role_id=db83c2f8e21945c3ab9642c620d29a23)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:13.748828 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d70d240-3d75-4256-93f9-5d56b22c51b9 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:13.750365 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e45382d-1027-4bd7-9ebc-9b1c3246aa9d None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:13.808430 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:13.826802 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 295/612] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Aug 29 22:29:13 2022] PUT /identity/v3/projects/ee84c3340f2641ad963655a7a212ef96/users/0fe5e8c4cb324ffb9d5b80f18d123727/roles/db83c2f8e21945c3ab9642c620d29a23 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:29:13.829221 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd51d4d-e7fc-49d0-95be-eeb467373e5e None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:13.830047 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd51d4d-e7fc-49d0-95be-eeb467373e5e None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:13.831233 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd51d4d-e7fc-49d0-95be-eeb467373e5e None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:13.838983 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 296/613] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:29:13.841445 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06947f2d-12e6-480b-9669-aca84b29fd5c None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:13.841950 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06947f2d-12e6-480b-9669-aca84b29fd5c None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:13.842423 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06947f2d-12e6-480b-9669-aca84b29fd5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:13.867002 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-06947f2d-12e6-480b-9669-aca84b29fd5c None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:13.868821 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-06947f2d-12e6-480b-9669-aca84b29fd5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:13.894610 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d70d240-3d75-4256-93f9-5d56b22c51b9 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:13.959149 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 297/614] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:29:13.978472 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c926d54f-0516-4312-ac8a-4ac5c8663e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:13.979102 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926d54f-0516-4312-ac8a-4ac5c8663e55 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:13.979379 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926d54f-0516-4312-ac8a-4ac5c8663e55 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:13.979650 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926d54f-0516-4312-ac8a-4ac5c8663e55 None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:13.980297 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c926d54f-0516-4312-ac8a-4ac5c8663e55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:13.980596 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c926d54f-0516-4312-ac8a-4ac5c8663e55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:13.982008 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c926d54f-0516-4312-ac8a-4ac5c8663e55 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:13.999641 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 318/615] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:29:14.007216 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06947f2d-12e6-480b-9669-aca84b29fd5c None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:14.072647 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:14.114657 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06947f2d-12e6-480b-9669-aca84b29fd5c None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:14.152599 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f24dd5-98b6-4672-9b4e-732ddf23ce56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:14.153284 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f24dd5-98b6-4672-9b4e-732ddf23ce56 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:14.153535 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f24dd5-98b6-4672-9b4e-732ddf23ce56 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:14.153805 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f24dd5-98b6-4672-9b4e-732ddf23ce56 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:14.161226 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f24dd5-98b6-4672-9b4e-732ddf23ce56 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:14.162670 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f24dd5-98b6-4672-9b4e-732ddf23ce56 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:14.175381 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-49f24dd5-98b6-4672-9b4e-732ddf23ce56 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 22:29:14.175381 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:14.175381 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:14.175381 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:14.175381 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:14.175381 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:14.175381 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:14.175381 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:14.175381 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:14.175381 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:14.175381 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:14.175381 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:14.175381 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:29:14.175381 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:29:14.175381 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:29:14.175381 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:29:14.176503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:14.176503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:14.176503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:29:14.176503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:14.176503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:29:14.176503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:14.176503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:14.176503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:14.176503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:14.176503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:14.176503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:14.176503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:14.176503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:14.176503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:14.176503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:14.176503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:14.176503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:14.176503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:14.177673 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:14.177673 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:14.177673 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:29:14.177673 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:29:14.177673 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:29:14.177673 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:29:14.177673 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:29:14.177673 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:29:14.177673 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 22:29:14.177673 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:14.177673 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 298/616] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Aug 29 22:29:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:14.182336 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c926d54f-0516-4312-ac8a-4ac5c8663e55 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:14.342203 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c926d54f-0516-4312-ac8a-4ac5c8663e55 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:14.355842 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:14.387456 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e558c28-7460-4843-adb2-75e148c26236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:14.388079 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e558c28-7460-4843-adb2-75e148c26236 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:14.388332 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e558c28-7460-4843-adb2-75e148c26236 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:14.388605 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e558c28-7460-4843-adb2-75e148c26236 None admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:14.389746 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e558c28-7460-4843-adb2-75e148c26236 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:14.390045 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e558c28-7460-4843-adb2-75e148c26236 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:14.394471 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f24dd5-98b6-4672-9b4e-732ddf23ce56 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:14.394617 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e558c28-7460-4843-adb2-75e148c26236 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:14.415958 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 319/617] 10.222.0.52 () {56 vars in 1135 bytes} [Mon Aug 29 22:29:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:14.421679 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e558c28-7460-4843-adb2-75e148c26236 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:14.547018 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e558c28-7460-4843-adb2-75e148c26236 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:14.560643 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f24dd5-98b6-4672-9b4e-732ddf23ce56 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:14.586070 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:14.589999 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f17f8c-2705-4d58-903c-935fccd8e444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:14.590820 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f17f8c-2705-4d58-903c-935fccd8e444 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:14.591254 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f17f8c-2705-4d58-903c-935fccd8e444 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:14.591680 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f17f8c-2705-4d58-903c-935fccd8e444 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:14.605633 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d14196b-5d10-4baf-8c07-dfc99c4639f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:14.605633 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14196b-5d10-4baf-8c07-dfc99c4639f8 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:14.605633 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14196b-5d10-4baf-8c07-dfc99c4639f8 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:14.605633 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14196b-5d10-4baf-8c07-dfc99c4639f8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:14.606090 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f17f8c-2705-4d58-903c-935fccd8e444 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:14.606090 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f17f8c-2705-4d58-903c-935fccd8e444 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:14.614780 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d14196b-5d10-4baf-8c07-dfc99c4639f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:29:14.619532 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-0d14196b-5d10-4baf-8c07-dfc99c4639f8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:29:14.619532 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:14.619532 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:14.619532 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:14.619532 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:14.619532 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:14.619532 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:14.619532 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:14.619532 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:14.619532 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:14.619532 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:14.619532 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:14.619532 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:29:14.619532 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:29:14.619532 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:29:14.619532 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:29:14.620938 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:29:14.620938 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:29:14.620938 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:29:14.620938 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:29:14.620938 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:14.620938 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:14.620938 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:14.620938 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:14.620938 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:14.620938 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:14.620938 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:14.620938 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:14.620938 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:14.620938 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:14.620938 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:14.620938 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:14.622103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:14.622103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:14.622103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:14.622103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:14.622103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:29:14.622103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:29:14.622103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:29:14.622103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:29:14.622103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:29:14.622103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:29:14.622103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:29:14.622103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:14.622103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 299/618] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Aug 29 22:29:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:14.622887 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d730643-556d-4df4-9a90-4c72fecea644 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:14.622887 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d730643-556d-4df4-9a90-4c72fecea644 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:14.622887 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d730643-556d-4df4-9a90-4c72fecea644 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:14.627898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-e0f17f8c-2705-4d58-903c-935fccd8e444 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:14.627898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:14.627898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:14.627898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:14.627898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:14.627898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:14.627898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:14.627898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:14.627898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:14.627898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:14.627898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:14.627898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:14.627898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:14.627898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:14.627898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:14.627898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:14.629141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:14.629141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:14.629141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:14.629141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:14.629141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:14.629141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:14.629141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:14.629141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:14.629141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:14.629141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:14.629141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:14.629141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:14.629141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:14.629141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:14.629141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:14.629141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:14.629141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:14.630103 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:14.630103 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:14.630103 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:14.630286 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 300/619] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:29:14.630286 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d730643-556d-4df4-9a90-4c72fecea644 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:14.631848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 320/620] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:29:14.633295 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c4e73-2f8a-4496-a3b6-c17951f40072 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:14.633786 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c4e73-2f8a-4496-a3b6-c17951f40072 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:14.634278 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c4e73-2f8a-4496-a3b6-c17951f40072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:14.653462 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-080c4e73-2f8a-4496-a3b6-c17951f40072 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:14.654995 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-080c4e73-2f8a-4496-a3b6-c17951f40072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:14.670717 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:14.735639 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d730643-556d-4df4-9a90-4c72fecea644 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:14.736385 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 321/621] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:29:14.738704 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080c4e73-2f8a-4496-a3b6-c17951f40072 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:14.771129 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a9857b-fe42-4cb8-a63f-ca22cb25785e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:14.771773 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9857b-fe42-4cb8-a63f-ca22cb25785e None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:14.772050 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9857b-fe42-4cb8-a63f-ca22cb25785e None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:14.772322 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9857b-fe42-4cb8-a63f-ca22cb25785e None admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:14.773062 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a9857b-fe42-4cb8-a63f-ca22cb25785e None admin] RBAC: Adding query filter params (name=service) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:14.773407 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a9857b-fe42-4cb8-a63f-ca22cb25785e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:14.774888 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a9857b-fe42-4cb8-a63f-ca22cb25785e None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:14.784428 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 301/622] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Aug 29 22:29:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:14.789845 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a9857b-fe42-4cb8-a63f-ca22cb25785e None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:14.831264 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:14.908911 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a9857b-fe42-4cb8-a63f-ca22cb25785e None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:14.935706 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080c4e73-2f8a-4496-a3b6-c17951f40072 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:14.944931 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f4fd2e-53c3-4f9b-9e4a-c90b3a157c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:14.944931 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4fd2e-53c3-4f9b-9e4a-c90b3a157c30 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:14.944931 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4fd2e-53c3-4f9b-9e4a-c90b3a157c30 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:14.944931 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4fd2e-53c3-4f9b-9e4a-c90b3a157c30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:14.952471 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f4fd2e-53c3-4f9b-9e4a-c90b3a157c30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:14.954558 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f4fd2e-53c3-4f9b-9e4a-c90b3a157c30 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:14.969093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-b4f4fd2e-53c3-4f9b-9e4a-c90b3a157c30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:14.969093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:14.969093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:14.969093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:14.969093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:14.969093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:14.969093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:14.969093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:14.969093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:14.969093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:14.969093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:14.969093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:14.969093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:14.969093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:14.969093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:14.969093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:14.970106 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:14.970106 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:14.970106 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:14.970106 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:14.970106 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:14.970106 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:14.970106 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:14.970106 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:14.970106 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:14.970106 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:14.970106 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:14.970106 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:14.970106 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:14.970106 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:14.970106 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:14.970106 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:14.970106 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:14.971326 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:14.971326 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:14.971326 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:14.971326 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 302/623] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:14.976527 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f4fd2e-53c3-4f9b-9e4a-c90b3a157c30 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:14.988644 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cb00b8-b762-4adc-89d4-1eb8c4e6128a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:14.989633 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb00b8-b762-4adc-89d4-1eb8c4e6128a None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:14.990109 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb00b8-b762-4adc-89d4-1eb8c4e6128a None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:14.990574 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb00b8-b762-4adc-89d4-1eb8c4e6128a None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:14.991432 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cb00b8-b762-4adc-89d4-1eb8c4e6128a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:14.991918 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cb00b8-b762-4adc-89d4-1eb8c4e6128a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:14.993517 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cb00b8-b762-4adc-89d4-1eb8c4e6128a None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:15.006953 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 322/624] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:15.012813 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cb00b8-b762-4adc-89d4-1eb8c4e6128a None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:15.028000 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:15.078798 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:15.099393 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f4fd2e-53c3-4f9b-9e4a-c90b3a157c30 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:15.130796 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cb00b8-b762-4adc-89d4-1eb8c4e6128a None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:15.136063 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81c5568-8f85-4059-8b09-2d6c05ce9fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:15.136734 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c5568-8f85-4059-8b09-2d6c05ce9fb4 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:15.137021 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c5568-8f85-4059-8b09-2d6c05ce9fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:15.137336 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c5568-8f85-4059-8b09-2d6c05ce9fb4 None admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:15.137985 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81c5568-8f85-4059-8b09-2d6c05ce9fb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:15.138284 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81c5568-8f85-4059-8b09-2d6c05ce9fb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:15.139646 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81c5568-8f85-4059-8b09-2d6c05ce9fb4 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:15.153495 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 303/625] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:15.159686 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81c5568-8f85-4059-8b09-2d6c05ce9fb4 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:15.171674 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928556fd-68ea-444a-8402-da1635e4f102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:15.172812 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928556fd-68ea-444a-8402-da1635e4f102 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:15.173296 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928556fd-68ea-444a-8402-da1635e4f102 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:15.173765 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928556fd-68ea-444a-8402-da1635e4f102 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:15.182180 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928556fd-68ea-444a-8402-da1635e4f102 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:15.184042 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928556fd-68ea-444a-8402-da1635e4f102 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:15.196788 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-928556fd-68ea-444a-8402-da1635e4f102 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:15.196788 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:15.196788 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:15.196788 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:15.196788 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:15.196788 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:15.196788 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:15.196788 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:15.196788 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:15.196788 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:15.196788 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:15.196788 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:15.196788 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:29:15.196788 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:29:15.196788 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:29:15.197734 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:29:15.197734 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:15.197734 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:15.197734 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:15.197734 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:15.197734 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:15.197734 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:15.197734 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:15.197734 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:15.197734 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:15.197734 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:15.197734 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:15.197734 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:15.197734 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:15.197734 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:15.197734 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:15.197734 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:15.198640 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:29:15.198640 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:29:15.198640 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:29:15.198640 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:29:15.198640 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:29:15.198640 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:29:15.198640 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:15.198640 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:15.201238 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 323/626] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:29:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:29:15.207707 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928556fd-68ea-444a-8402-da1635e4f102 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:15.279289 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81c5568-8f85-4059-8b09-2d6c05ce9fb4 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:15.284045 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:15.317606 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aec4c40-5464-4dfd-a3a3-461b0a20da52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:15.318521 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec4c40-5464-4dfd-a3a3-461b0a20da52 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:15.318994 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec4c40-5464-4dfd-a3a3-461b0a20da52 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:15.319873 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec4c40-5464-4dfd-a3a3-461b0a20da52 None admin] PATH_INFO: `/v3/users/glance` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:15.321667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928556fd-68ea-444a-8402-da1635e4f102 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:15.328712 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aec4c40-5464-4dfd-a3a3-461b0a20da52 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:15.331232 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aec4c40-5464-4dfd-a3a3-461b0a20da52 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:15.344588 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-6aec4c40-5464-4dfd-a3a3-461b0a20da52 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 29 22:29:15.344588 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:15.344588 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:15.344588 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:15.344588 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:15.344588 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:15.344588 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:15.344588 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:15.344588 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:15.344588 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:15.344588 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:15.344588 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:15.344588 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:29:15.344588 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:29:15.344588 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:29:15.344588 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:29:15.345598 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:15.345598 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:15.345598 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:29:15.345598 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:15.345598 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:29:15.345598 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:15.345598 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:15.345598 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:15.345598 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:15.345598 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:15.345598 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:15.345598 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:15.345598 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:15.345598 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:15.345598 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:15.345598 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:15.345598 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:15.345598 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:15.346772 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:15.346772 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:15.346772 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:29:15.346772 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:29:15.346772 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:29:15.346772 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:29:15.346772 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:29:15.346772 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:29:15.346772 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 29 22:29:15.346772 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:15.353526 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 304/627] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:29:15 2022] GET /identity/v3/users/glance => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:15.354638 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aec4c40-5464-4dfd-a3a3-461b0a20da52 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:15.357730 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:15.360602 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3072855-e522-4d26-bd45-c2d69f3578a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:15.361375 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3072855-e522-4d26-bd45-c2d69f3578a9 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:15.361747 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3072855-e522-4d26-bd45-c2d69f3578a9 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:15.362120 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3072855-e522-4d26-bd45-c2d69f3578a9 None admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:15.362909 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3072855-e522-4d26-bd45-c2d69f3578a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:15.363287 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3072855-e522-4d26-bd45-c2d69f3578a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:15.364800 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3072855-e522-4d26-bd45-c2d69f3578a9 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:15.378356 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 324/628] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Aug 29 22:29:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:29:15.379298 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3072855-e522-4d26-bd45-c2d69f3578a9 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:15.466675 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aec4c40-5464-4dfd-a3a3-461b0a20da52 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:15.487611 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3072855-e522-4d26-bd45-c2d69f3578a9 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:15.502700 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b41a280-46c0-4437-b00d-bd6c38f74cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:15.503353 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41a280-46c0-4437-b00d-bd6c38f74cba None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:15.503627 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41a280-46c0-4437-b00d-bd6c38f74cba None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:15.503904 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41a280-46c0-4437-b00d-bd6c38f74cba None admin] PATH_INFO: `/v3/roles/service` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:15.514530 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b41a280-46c0-4437-b00d-bd6c38f74cba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:29:15.520045 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-8b41a280-46c0-4437-b00d-bd6c38f74cba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:29:15.520045 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:15.520045 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:15.520045 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:15.520045 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:15.520045 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:15.520045 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:15.520045 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:15.520045 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:15.520045 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:15.520045 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:15.520045 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:15.520045 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:29:15.520045 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:29:15.520045 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:29:15.520045 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:29:15.521310 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:29:15.521310 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:29:15.521310 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:29:15.521310 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:29:15.521310 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:15.521310 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:15.521310 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:15.521310 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:15.521310 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:15.521310 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:15.521310 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:15.521310 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:15.521310 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:15.521310 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:15.521310 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:15.521310 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:15.522314 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:15.522314 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:15.522314 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:15.522314 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:15.522314 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:29:15.522314 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:29:15.522314 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:29:15.522314 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:29:15.522314 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:29:15.522314 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:29:15.522314 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:29:15.522314 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:15.522314 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 305/629] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Aug 29 22:29:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:15.523149 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e92f2d-a1f8-4de4-afb8-caa9ab71ad8d None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:15.523149 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e92f2d-a1f8-4de4-afb8-caa9ab71ad8d None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:15.523149 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e92f2d-a1f8-4de4-afb8-caa9ab71ad8d None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:15.526099 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 306/630] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:29:15.526099 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e92f2d-a1f8-4de4-afb8-caa9ab71ad8d None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:15.531871 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:15.546858 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da00a773-5b4b-4375-9a92-2f3b9cbd9bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:15.550753 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00a773-5b4b-4375-9a92-2f3b9cbd9bf9 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:15.550753 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00a773-5b4b-4375-9a92-2f3b9cbd9bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:15.550753 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00a773-5b4b-4375-9a92-2f3b9cbd9bf9 None admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:15.550753 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da00a773-5b4b-4375-9a92-2f3b9cbd9bf9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:15.550753 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da00a773-5b4b-4375-9a92-2f3b9cbd9bf9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:15.551858 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da00a773-5b4b-4375-9a92-2f3b9cbd9bf9 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:15.571110 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 325/631] 10.222.0.52 () {56 vars in 1137 bytes} [Mon Aug 29 22:29:15 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:15.571991 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da00a773-5b4b-4375-9a92-2f3b9cbd9bf9 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:15.601041 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:15.647678 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e92f2d-a1f8-4de4-afb8-caa9ab71ad8d None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:15.683851 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb768c0-9313-469e-812b-253d8c7ed786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:15.684507 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb768c0-9313-469e-812b-253d8c7ed786 None admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:15.684823 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb768c0-9313-469e-812b-253d8c7ed786 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:15.685122 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb768c0-9313-469e-812b-253d8c7ed786 None admin] PATH_INFO: `/v3/projects/ee84c3340f2641ad963655a7a212ef96/users/57c175ac90e545358746176972ff39a3/roles/db83c2f8e21945c3ab9642c620d29a23` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:15.685336 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da00a773-5b4b-4375-9a92-2f3b9cbd9bf9 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:15.701118 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb768c0-9313-469e-812b-253d8c7ed786 None admin] RBAC: Authorizing `identity:create_grant(project_id=ee84c3340f2641ad963655a7a212ef96, user_id=57c175ac90e545358746176972ff39a3, role_id=db83c2f8e21945c3ab9642c620d29a23)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:15.702566 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb768c0-9313-469e-812b-253d8c7ed786 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:15.723289 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576a9547-a0b4-429d-bd53-7b507cc94388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:15.723817 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a9547-a0b4-429d-bd53-7b507cc94388 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:15.724097 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a9547-a0b4-429d-bd53-7b507cc94388 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:15.724370 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a9547-a0b4-429d-bd53-7b507cc94388 None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:15.725070 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576a9547-a0b4-429d-bd53-7b507cc94388 None admin] RBAC: Adding query filter params (name=service) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:15.725377 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576a9547-a0b4-429d-bd53-7b507cc94388 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:15.726757 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576a9547-a0b4-429d-bd53-7b507cc94388 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:15.735561 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 326/632] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Aug 29 22:29:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:15.737820 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b691e1-2bfb-42e7-815e-5a44f1e445b4 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:15.739596 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b691e1-2bfb-42e7-815e-5a44f1e445b4 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:15.739596 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b691e1-2bfb-42e7-815e-5a44f1e445b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:15.756476 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-f7b691e1-2bfb-42e7-815e-5a44f1e445b4 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:15.757698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-f7b691e1-2bfb-42e7-815e-5a44f1e445b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:15.785145 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 307/633] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Aug 29 22:29:15 2022] PUT /identity/v3/projects/ee84c3340f2641ad963655a7a212ef96/users/57c175ac90e545358746176972ff39a3/roles/db83c2f8e21945c3ab9642c620d29a23 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:29:15.786006 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb768c0-9313-469e-812b-253d8c7ed786 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:15.818257 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:15.848090 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 327/634] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:29:15.849378 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b691e1-2bfb-42e7-815e-5a44f1e445b4 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:15.972084 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb768c0-9313-469e-812b-253d8c7ed786 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:15.993816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:16.018673 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610fd3d5-6aa9-4800-8a1d-8ff589a3361b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:16.019553 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610fd3d5-6aa9-4800-8a1d-8ff589a3361b None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:16.019834 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610fd3d5-6aa9-4800-8a1d-8ff589a3361b None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:16.020108 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610fd3d5-6aa9-4800-8a1d-8ff589a3361b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:16.045449 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b691e1-2bfb-42e7-815e-5a44f1e445b4 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:16.117692 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa21a738-ec6f-4887-8dab-c66133b8e723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:16.118735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21a738-ec6f-4887-8dab-c66133b8e723 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:16.119211 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21a738-ec6f-4887-8dab-c66133b8e723 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:16.119686 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21a738-ec6f-4887-8dab-c66133b8e723 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:16.140911 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa21a738-ec6f-4887-8dab-c66133b8e723 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:16.143443 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa21a738-ec6f-4887-8dab-c66133b8e723 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:16.144366 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610fd3d5-6aa9-4800-8a1d-8ff589a3361b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:16.146029 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610fd3d5-6aa9-4800-8a1d-8ff589a3361b None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:16.170885 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-aa21a738-ec6f-4887-8dab-c66133b8e723 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:16.170885 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:16.170885 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:16.170885 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:16.170885 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:16.170885 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:16.170885 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:16.170885 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:16.170885 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:16.170885 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:16.170885 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:16.170885 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:16.170885 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:16.170885 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:16.170885 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:16.170885 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:16.171897 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:16.171897 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:16.171897 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:16.171897 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:16.171897 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:16.171897 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:16.171897 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:16.171897 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:16.171897 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:16.171897 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:16.171897 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:16.171897 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:16.171897 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:16.171897 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:16.171897 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:16.171897 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:16.171897 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:16.173522 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:16.173522 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:16.173522 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:16.173522 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 328/635] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:16.173522 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa21a738-ec6f-4887-8dab-c66133b8e723 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:16.204896 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:16.263283 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 308/636] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:29:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:29:16.264784 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e08b4-1dd0-41f5-b98e-b6c66fe6c159 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:16.265066 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e08b4-1dd0-41f5-b98e-b6c66fe6c159 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:16.265337 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e08b4-1dd0-41f5-b98e-b6c66fe6c159 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:16.266717 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 309/637] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:29:16.267222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67e08b4-1dd0-41f5-b98e-b6c66fe6c159 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:16.284670 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:16.291204 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa21a738-ec6f-4887-8dab-c66133b8e723 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:16.332176 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3747db-8fa4-4689-a48e-6b63278380e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:16.332873 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3747db-8fa4-4689-a48e-6b63278380e2 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:16.333160 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3747db-8fa4-4689-a48e-6b63278380e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:16.333435 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3747db-8fa4-4689-a48e-6b63278380e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:16.344541 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3747db-8fa4-4689-a48e-6b63278380e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:29:16.351148 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-fa3747db-8fa4-4689-a48e-6b63278380e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:29:16.351148 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:16.351148 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:16.351148 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:16.351148 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:16.351148 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:16.351148 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:16.351148 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:16.351148 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:16.351148 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:16.351148 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:16.351148 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:16.351148 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:29:16.351148 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:29:16.351148 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:29:16.351148 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:29:16.352113 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:29:16.352113 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:29:16.352113 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:29:16.352113 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:29:16.352113 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:16.352113 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:16.352113 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:16.352113 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:16.352113 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:16.352113 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:16.352113 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:16.352113 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:16.352113 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:16.352113 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:16.352113 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:16.352113 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:16.353416 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:16.353416 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:16.353416 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:16.353416 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:16.353416 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:29:16.353416 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:29:16.353416 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:29:16.353416 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:29:16.353416 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:29:16.353416 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:29:16.353416 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:29:16.353416 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:16.353416 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 329/638] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Aug 29 22:29:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:16.354137 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3747db-8fa4-4689-a48e-6b63278380e2 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:16.388507 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67e08b4-1dd0-41f5-b98e-b6c66fe6c159 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:16.428336 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8503d78f-955c-4621-87e5-3eb03cf5c8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:16.429395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8503d78f-955c-4621-87e5-3eb03cf5c8b5 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:16.429395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8503d78f-955c-4621-87e5-3eb03cf5c8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:16.429691 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8503d78f-955c-4621-87e5-3eb03cf5c8b5 None admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:16.430104 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8503d78f-955c-4621-87e5-3eb03cf5c8b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:16.430403 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8503d78f-955c-4621-87e5-3eb03cf5c8b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:16.431760 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8503d78f-955c-4621-87e5-3eb03cf5c8b5 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:16.448973 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 310/639] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:16.450425 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5302cbb0-1546-4313-a6a6-0bdf9849b46a None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:16.450714 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5302cbb0-1546-4313-a6a6-0bdf9849b46a None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:16.450989 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5302cbb0-1546-4313-a6a6-0bdf9849b46a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:16.453040 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:16.463510 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3747db-8fa4-4689-a48e-6b63278380e2 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:16.469315 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-5302cbb0-1546-4313-a6a6-0bdf9849b46a None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:16.470750 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-5302cbb0-1546-4313-a6a6-0bdf9849b46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:16.501728 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e120b457-930a-482e-b83b-94c992ebd24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:16.502973 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e120b457-930a-482e-b83b-94c992ebd24a None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:16.503515 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e120b457-930a-482e-b83b-94c992ebd24a None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:16.503515 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e120b457-930a-482e-b83b-94c992ebd24a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:16.514488 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e120b457-930a-482e-b83b-94c992ebd24a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:16.515861 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e120b457-930a-482e-b83b-94c992ebd24a None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:16.530704 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-e120b457-930a-482e-b83b-94c992ebd24a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:16.530704 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:16.530704 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:16.530704 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:16.530704 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:16.530704 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:16.530704 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:16.530704 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:16.530704 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:16.530704 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:16.530704 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:16.530704 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:16.530704 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:16.530704 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:16.530704 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:16.530704 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:16.531703 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:16.531703 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:16.531703 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:16.531703 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:16.531703 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:16.531703 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:16.531703 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:16.531703 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:16.531703 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:16.531703 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:16.531703 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:16.531703 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:16.531703 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:16.531703 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:16.531703 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:16.531703 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:16.531703 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:16.532616 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:16.532616 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:16.532616 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:16.532616 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 330/640] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:16.532616 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e120b457-930a-482e-b83b-94c992ebd24a None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:16.553723 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 311/641] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:29:16.554895 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5302cbb0-1546-4313-a6a6-0bdf9849b46a None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:16.557451 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:16.664631 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5302cbb0-1546-4313-a6a6-0bdf9849b46a None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:16.681780 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e120b457-930a-482e-b83b-94c992ebd24a None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:16.701806 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f575e-aadf-4089-b30f-66d936a733bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:16.702521 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f575e-aadf-4089-b30f-66d936a733bb None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:16.702875 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f575e-aadf-4089-b30f-66d936a733bb None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:16.703219 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f575e-aadf-4089-b30f-66d936a733bb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:16.711644 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900f575e-aadf-4089-b30f-66d936a733bb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:16.713268 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900f575e-aadf-4089-b30f-66d936a733bb None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:16.726373 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-900f575e-aadf-4089-b30f-66d936a733bb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 22:29:16.726373 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:16.726373 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:16.726373 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:16.726373 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:16.726373 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:16.726373 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:16.726373 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:16.726373 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:16.726373 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:16.726373 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:16.726373 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:16.726373 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:29:16.726373 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:29:16.726373 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:29:16.726373 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:29:16.727382 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:16.727382 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:16.727382 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:29:16.727382 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:16.727382 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:29:16.727382 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:16.727382 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:16.727382 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:16.727382 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:16.727382 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:16.727382 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:16.727382 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:16.727382 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:16.727382 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:16.727382 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:16.727382 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:16.727382 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:16.727382 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:16.728303 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:16.728303 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:16.728303 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:29:16.728303 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:29:16.728303 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:29:16.728303 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:29:16.728303 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:29:16.728303 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:29:16.728303 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 22:29:16.728303 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:16.728303 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 312/642] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:29:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:16.728303 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f575e-aadf-4089-b30f-66d936a733bb None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:16.728994 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2516dc9d-7aaf-4c27-90c2-29d4b5005102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:16.734654 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516dc9d-7aaf-4c27-90c2-29d4b5005102 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:16.734654 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516dc9d-7aaf-4c27-90c2-29d4b5005102 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:16.734654 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516dc9d-7aaf-4c27-90c2-29d4b5005102 None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:16.734654 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2516dc9d-7aaf-4c27-90c2-29d4b5005102 None admin] RBAC: Adding query filter params (name=service) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:16.734654 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2516dc9d-7aaf-4c27-90c2-29d4b5005102 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:16.734654 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2516dc9d-7aaf-4c27-90c2-29d4b5005102 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:16.749098 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 331/643] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Aug 29 22:29:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:16.749977 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2516dc9d-7aaf-4c27-90c2-29d4b5005102 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:16.754664 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:16.813031 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:16.856025 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f575e-aadf-4089-b30f-66d936a733bb None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:16.865206 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2516dc9d-7aaf-4c27-90c2-29d4b5005102 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:16.894672 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d25ad-7d3c-4548-a9d6-cfdda073e218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:16.895272 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d25ad-7d3c-4548-a9d6-cfdda073e218 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:16.895548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d25ad-7d3c-4548-a9d6-cfdda073e218 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:16.895817 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d25ad-7d3c-4548-a9d6-cfdda073e218 None admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:16.896541 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1d25ad-7d3c-4548-a9d6-cfdda073e218 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:16.896811 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1d25ad-7d3c-4548-a9d6-cfdda073e218 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:16.898131 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1d25ad-7d3c-4548-a9d6-cfdda073e218 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:16.905443 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f03d59-3765-4199-bb67-8ae35f02c711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:16.906053 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f03d59-3765-4199-bb67-8ae35f02c711 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:16.906335 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f03d59-3765-4199-bb67-8ae35f02c711 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:16.906608 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f03d59-3765-4199-bb67-8ae35f02c711 None admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:16.907341 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f03d59-3765-4199-bb67-8ae35f02c711 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:16.907643 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f03d59-3765-4199-bb67-8ae35f02c711 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:16.909271 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f03d59-3765-4199-bb67-8ae35f02c711 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:16.910957 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 313/644] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:16.911769 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d25ad-7d3c-4548-a9d6-cfdda073e218 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:16.936809 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 332/645] 10.222.0.52 () {56 vars in 1137 bytes} [Mon Aug 29 22:29:16 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:16.936809 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f03d59-3765-4199-bb67-8ae35f02c711 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:17.004671 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:17.035139 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d25ad-7d3c-4548-a9d6-cfdda073e218 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:17.053390 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f03d59-3765-4199-bb67-8ae35f02c711 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:17.066007 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:17.079835 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5518e3-1aad-4377-adb5-d3bc64eaa4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:17.082883 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5518e3-1aad-4377-adb5-d3bc64eaa4dd None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:17.082883 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5518e3-1aad-4377-adb5-d3bc64eaa4dd None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:17.082883 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5518e3-1aad-4377-adb5-d3bc64eaa4dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:17.090465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94888c3-b19f-4a29-ab27-da9dc68bd8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:17.091093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94888c3-b19f-4a29-ab27-da9dc68bd8cb None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:17.091368 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94888c3-b19f-4a29-ab27-da9dc68bd8cb None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:17.091698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94888c3-b19f-4a29-ab27-da9dc68bd8cb None admin] PATH_INFO: `/v3/roles/service` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:17.092196 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5518e3-1aad-4377-adb5-d3bc64eaa4dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:17.094876 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5518e3-1aad-4377-adb5-d3bc64eaa4dd None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:17.103885 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94888c3-b19f-4a29-ab27-da9dc68bd8cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:29:17.109120 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-d94888c3-b19f-4a29-ab27-da9dc68bd8cb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:29:17.109120 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:17.109120 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:17.109120 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:17.109120 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:17.109120 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:17.109120 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:17.109120 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:17.109120 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:17.109120 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:17.109120 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:17.109120 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:17.109120 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:29:17.109120 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:29:17.109120 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:29:17.109120 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:29:17.118093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:29:17.118093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:29:17.118093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:29:17.118093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:29:17.118093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:17.118093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:17.118093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:17.118093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:17.118093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:17.118093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:17.118093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:17.118093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:17.118093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:17.118093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:17.118093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:17.118093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:17.119921 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:17.119921 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:17.119921 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:17.119921 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:17.119921 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:29:17.119921 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:29:17.119921 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:29:17.119921 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:29:17.119921 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:29:17.119921 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:29:17.119921 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:29:17.119921 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:17.119921 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 333/646] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Aug 29 22:29:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:17.119921 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94888c3-b19f-4a29-ab27-da9dc68bd8cb None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:17.120891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-fa5518e3-1aad-4377-adb5-d3bc64eaa4dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:17.120891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:17.120891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:17.120891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:17.120891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:17.120891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:17.120891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:17.120891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:17.120891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:17.120891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:17.120891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:17.120891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:17.120891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:17.120891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:17.120891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:17.120891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:17.121967 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:17.121967 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:17.121967 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:17.121967 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:17.121967 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:17.121967 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:17.121967 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:17.121967 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:17.121967 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:17.121967 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:17.121967 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:17.121967 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:17.121967 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:17.121967 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:17.121967 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:17.121967 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:17.121967 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:17.123072 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:17.123072 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:17.123072 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:17.124496 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 314/647] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:17.125799 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5518e3-1aad-4377-adb5-d3bc64eaa4dd None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:17.239678 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94888c3-b19f-4a29-ab27-da9dc68bd8cb None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:17.241252 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5518e3-1aad-4377-adb5-d3bc64eaa4dd None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:17.263443 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:17.276001 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d83889-8a00-465c-a29b-d20e26f92a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:17.276628 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d83889-8a00-465c-a29b-d20e26f92a8c None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:17.276932 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d83889-8a00-465c-a29b-d20e26f92a8c None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:17.278239 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d83889-8a00-465c-a29b-d20e26f92a8c None admin] PATH_INFO: `/v3/projects/service` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:17.280013 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d80a76-4a17-4634-afe0-dd55b34e0c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:17.280800 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d80a76-4a17-4634-afe0-dd55b34e0c3c None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:17.281224 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d80a76-4a17-4634-afe0-dd55b34e0c3c None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:17.281657 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d80a76-4a17-4634-afe0-dd55b34e0c3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:17.285301 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d83889-8a00-465c-a29b-d20e26f92a8c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:17.286809 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d83889-8a00-465c-a29b-d20e26f92a8c None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:17.298314 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-67d83889-8a00-465c-a29b-d20e26f92a8c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:17.298314 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:17.298314 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:17.298314 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:17.298314 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:17.298314 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:17.298314 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:17.298314 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:17.298314 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:17.298314 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:17.298314 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:17.298314 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:17.298314 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:29:17.298314 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:29:17.298314 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:29:17.299924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:29:17.299924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:17.299924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:17.299924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:17.299924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:17.299924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:17.299924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:17.299924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:17.299924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:17.299924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:17.299924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:17.299924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:17.299924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:17.299924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:17.299924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:17.299924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:17.299924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:17.300878 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:29:17.300878 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:29:17.300878 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:29:17.300878 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:29:17.300878 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:29:17.300878 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:29:17.300878 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:17.300878 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:17.300878 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 334/648] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:29:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:17.301369 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d83889-8a00-465c-a29b-d20e26f92a8c None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:17.340031 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:17.369082 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d80a76-4a17-4634-afe0-dd55b34e0c3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:17.370745 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d80a76-4a17-4634-afe0-dd55b34e0c3c None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:17.406512 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d83889-8a00-465c-a29b-d20e26f92a8c None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:17.443493 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5d43e-7516-4884-ab7f-3b3b9fcc6a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:17.444101 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5d43e-7516-4884-ab7f-3b3b9fcc6a4a None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:17.444379 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5d43e-7516-4884-ab7f-3b3b9fcc6a4a None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:17.444647 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5d43e-7516-4884-ab7f-3b3b9fcc6a4a None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:17.445348 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d5d43e-7516-4884-ab7f-3b3b9fcc6a4a None admin] RBAC: Adding query filter params (name=service) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:17.445632 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d5d43e-7516-4884-ab7f-3b3b9fcc6a4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:17.446987 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d5d43e-7516-4884-ab7f-3b3b9fcc6a4a None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:17.458639 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 335/649] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Aug 29 22:29:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:17.464151 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5d43e-7516-4884-ab7f-3b3b9fcc6a4a None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:17.482385 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 315/650] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:29:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:29:17.483160 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d80a76-4a17-4634-afe0-dd55b34e0c3c None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:17.517141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:17.568920 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:17.590485 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5d43e-7516-4884-ab7f-3b3b9fcc6a4a None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:17.596773 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d80a76-4a17-4634-afe0-dd55b34e0c3c None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:17.630048 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d66b84-5c7e-4baa-b657-9ce0f408b7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:17.631004 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d66b84-5c7e-4baa-b657-9ce0f408b7e1 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:17.631004 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d66b84-5c7e-4baa-b657-9ce0f408b7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:17.631298 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d66b84-5c7e-4baa-b657-9ce0f408b7e1 None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:17.632053 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d66b84-5c7e-4baa-b657-9ce0f408b7e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:17.632159 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d66b84-5c7e-4baa-b657-9ce0f408b7e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:17.633558 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d66b84-5c7e-4baa-b657-9ce0f408b7e1 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:17.634671 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e495a6a-7b0b-4ee1-8d92-51798f549f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:17.635335 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e495a6a-7b0b-4ee1-8d92-51798f549f5c None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:17.635614 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e495a6a-7b0b-4ee1-8d92-51798f549f5c None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:17.635882 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e495a6a-7b0b-4ee1-8d92-51798f549f5c None admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:17.636577 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e495a6a-7b0b-4ee1-8d92-51798f549f5c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:17.636911 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e495a6a-7b0b-4ee1-8d92-51798f549f5c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:17.638318 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e495a6a-7b0b-4ee1-8d92-51798f549f5c None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:17.647302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 336/651] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:17.647948 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d66b84-5c7e-4baa-b657-9ce0f408b7e1 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:17.660981 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 316/652] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Aug 29 22:29:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:17.660981 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e495a6a-7b0b-4ee1-8d92-51798f549f5c None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:17.762120 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d66b84-5c7e-4baa-b657-9ce0f408b7e1 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:17.765404 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:17.765522 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e495a6a-7b0b-4ee1-8d92-51798f549f5c None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:17.801182 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e3ad5c-07df-44b7-b727-a4e4d4383f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:17.802367 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e3ad5c-07df-44b7-b727-a4e4d4383f67 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:17.802728 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e3ad5c-07df-44b7-b727-a4e4d4383f67 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:17.802920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e3ad5c-07df-44b7-b727-a4e4d4383f67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:17.806316 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c697f22-a209-4e9e-a380-f290f8b9efde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:17.806947 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c697f22-a209-4e9e-a380-f290f8b9efde None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:17.807228 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c697f22-a209-4e9e-a380-f290f8b9efde None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:17.807502 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c697f22-a209-4e9e-a380-f290f8b9efde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:17.809469 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:17.813739 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e3ad5c-07df-44b7-b727-a4e4d4383f67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:17.815816 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e3ad5c-07df-44b7-b727-a4e4d4383f67 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:17.820384 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c697f22-a209-4e9e-a380-f290f8b9efde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:17.822009 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c697f22-a209-4e9e-a380-f290f8b9efde None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:17.833225 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-e1e3ad5c-07df-44b7-b727-a4e4d4383f67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:17.833225 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:17.833225 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:17.833225 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:17.833225 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:17.833225 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:17.833225 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:17.833225 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:17.833225 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:17.833225 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:17.833225 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:17.833225 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:17.833225 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:17.833225 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:17.833225 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:17.833225 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:17.834205 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:17.834205 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:17.834205 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:17.834205 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:17.834205 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:17.834205 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:17.834205 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:17.834205 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:17.834205 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:17.834205 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:17.834205 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:17.834205 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:17.834205 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:17.834205 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:17.834205 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:17.834205 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:17.834205 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:17.840465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:17.840465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:17.840465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:17.840465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 337/653] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:17.840465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e3ad5c-07df-44b7-b727-a4e4d4383f67 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:17.840834 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-4c697f22-a209-4e9e-a380-f290f8b9efde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:17.840834 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:17.840834 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:17.840834 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:17.840834 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:17.840834 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:17.840834 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:17.840834 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:17.840834 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:17.840834 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:17.840834 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:17.840834 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:17.840834 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:17.840834 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:17.840834 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:17.840834 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:17.841662 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:17.841662 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:17.841662 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:17.841662 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:17.841662 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:17.841662 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:17.841662 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:17.841662 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:17.841662 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:17.841662 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:17.841662 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:17.841662 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:17.841662 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:17.841662 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:17.841662 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:17.841662 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:17.841662 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:17.842855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:17.842855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:17.842855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:17.842855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 317/654] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:17.842855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c697f22-a209-4e9e-a380-f290f8b9efde None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:17.944967 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c697f22-a209-4e9e-a380-f290f8b9efde None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:17.961821 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e3ad5c-07df-44b7-b727-a4e4d4383f67 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:17.982649 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc55536d-e2ce-4060-b19c-204e116a01cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:17.983261 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55536d-e2ce-4060-b19c-204e116a01cf None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:17.983544 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55536d-e2ce-4060-b19c-204e116a01cf None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:17.983811 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55536d-e2ce-4060-b19c-204e116a01cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:17.984539 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc55536d-e2ce-4060-b19c-204e116a01cf None admin] RBAC: Adding query filter params (role.id=db83c2f8e21945c3ab9642c620d29a23, scope.project.id=ee84c3340f2641ad963655a7a212ef96, user.id=9c2569e7cd774199aa5750fd6a850060) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:17.984898 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc55536d-e2ce-4060-b19c-204e116a01cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:17.986986 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc55536d-e2ce-4060-b19c-204e116a01cf None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:17.993210 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 318/655] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:29:17 2022] GET /identity/v3/role_assignments?user.id=9c2569e7cd774199aa5750fd6a850060&scope.project.id=ee84c3340f2641ad963655a7a212ef96&role.id=db83c2f8e21945c3ab9642c620d29a23 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:17.994002 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc55536d-e2ce-4060-b19c-204e116a01cf None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:17.998939 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a66959-868a-4efb-b778-0b8c35412936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:17.999593 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a66959-868a-4efb-b778-0b8c35412936 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:17.999878 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a66959-868a-4efb-b778-0b8c35412936 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:18.000157 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a66959-868a-4efb-b778-0b8c35412936 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:18.013878 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a66959-868a-4efb-b778-0b8c35412936 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:18.015385 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a66959-868a-4efb-b778-0b8c35412936 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:18.029093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-93a66959-868a-4efb-b778-0b8c35412936 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 22:29:18.029093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:18.029093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:18.029093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:18.029093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:18.029093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:18.029093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:18.029093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:18.029093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:18.029093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:18.029093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:18.029093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:18.029093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:29:18.029093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:29:18.029093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:29:18.029093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:29:18.030292 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:18.030292 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:18.030292 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:29:18.030292 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:18.030292 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:29:18.030292 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:18.030292 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:18.030292 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:18.030292 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:18.030292 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:18.030292 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:18.030292 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:18.030292 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:18.030292 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:18.030292 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:18.030292 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:18.030292 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:18.030292 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:18.031264 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:18.031264 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:18.031264 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:29:18.031264 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:29:18.031264 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:29:18.031264 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:29:18.031264 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:29:18.031264 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:29:18.031264 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 22:29:18.031264 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:18.031264 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 338/656] 10.222.0.52 () {56 vars in 1093 bytes} [Mon Aug 29 22:29:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:18.031264 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a66959-868a-4efb-b778-0b8c35412936 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:18.054354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:18.064082 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:18.121317 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc55536d-e2ce-4060-b19c-204e116a01cf None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:18.152735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a66959-868a-4efb-b778-0b8c35412936 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:18.162336 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf283e93-f86d-416a-b224-509e187f5141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:18.163423 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf283e93-f86d-416a-b224-509e187f5141 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:18.163721 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf283e93-f86d-416a-b224-509e187f5141 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:18.163997 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf283e93-f86d-416a-b224-509e187f5141 None admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:18.164653 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf283e93-f86d-416a-b224-509e187f5141 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:18.165718 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf283e93-f86d-416a-b224-509e187f5141 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:18.167077 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf283e93-f86d-416a-b224-509e187f5141 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:18.181774 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 319/657] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:18.183037 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b81e9e-f70d-47ce-a0dd-fce13173fb9a None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:18.183324 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b81e9e-f70d-47ce-a0dd-fce13173fb9a None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:18.183604 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b81e9e-f70d-47ce-a0dd-fce13173fb9a None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:18.185056 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 320/658] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:29:18.185764 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b81e9e-f70d-47ce-a0dd-fce13173fb9a None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:18.196265 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be884f3a-a541-4cd2-b71d-5cd4a8096cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:18.197016 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be884f3a-a541-4cd2-b71d-5cd4a8096cf6 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:18.197367 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be884f3a-a541-4cd2-b71d-5cd4a8096cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:18.197650 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be884f3a-a541-4cd2-b71d-5cd4a8096cf6 None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:18.198299 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be884f3a-a541-4cd2-b71d-5cd4a8096cf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:18.198606 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be884f3a-a541-4cd2-b71d-5cd4a8096cf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:18.199975 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be884f3a-a541-4cd2-b71d-5cd4a8096cf6 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:18.213618 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 339/659] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:18.218968 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be884f3a-a541-4cd2-b71d-5cd4a8096cf6 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:18.306479 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b81e9e-f70d-47ce-a0dd-fce13173fb9a None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:18.310072 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:18.319484 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:18.338254 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be884f3a-a541-4cd2-b71d-5cd4a8096cf6 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:18.345364 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c50cd3-8f17-4262-8f2f-5aed9ebc4706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:18.346559 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c50cd3-8f17-4262-8f2f-5aed9ebc4706 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:18.346837 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c50cd3-8f17-4262-8f2f-5aed9ebc4706 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:18.347341 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c50cd3-8f17-4262-8f2f-5aed9ebc4706 None admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:18.347938 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c50cd3-8f17-4262-8f2f-5aed9ebc4706 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:18.348238 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c50cd3-8f17-4262-8f2f-5aed9ebc4706 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:18.349769 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c50cd3-8f17-4262-8f2f-5aed9ebc4706 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:18.368121 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 321/660] 10.222.0.52 () {56 vars in 1133 bytes} [Mon Aug 29 22:29:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:18.369778 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a43df7-d013-4ad9-92e2-b98f779155ff None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:18.370128 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a43df7-d013-4ad9-92e2-b98f779155ff None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:18.370465 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a43df7-d013-4ad9-92e2-b98f779155ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:18.385995 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bca51e-2542-4fec-a74c-483765a66e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:18.386908 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bca51e-2542-4fec-a74c-483765a66e89 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:18.387335 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bca51e-2542-4fec-a74c-483765a66e89 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:18.387802 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bca51e-2542-4fec-a74c-483765a66e89 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:18.389618 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-80a43df7-d013-4ad9-92e2-b98f779155ff None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:18.391706 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-80a43df7-d013-4ad9-92e2-b98f779155ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:18.397581 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bca51e-2542-4fec-a74c-483765a66e89 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:18.399275 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bca51e-2542-4fec-a74c-483765a66e89 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:18.412924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-09bca51e-2542-4fec-a74c-483765a66e89 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 22:29:18.412924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:18.412924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:18.412924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:18.412924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:18.412924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:18.412924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:18.412924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:18.412924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:18.412924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:18.412924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:18.412924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:18.412924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:29:18.412924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:29:18.412924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:29:18.412924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:29:18.414105 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:18.414105 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:18.414105 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:29:18.414105 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:18.414105 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:29:18.414105 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:18.414105 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:18.414105 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:18.414105 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:18.414105 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:18.414105 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:18.414105 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:18.414105 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:18.414105 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:18.414105 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:18.414105 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:18.414105 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:18.414105 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:18.415210 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:18.415210 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:18.415210 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:29:18.415210 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:29:18.415210 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:29:18.415210 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:29:18.415210 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:29:18.415210 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:29:18.415210 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 22:29:18.415210 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:18.415210 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 340/661] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Aug 29 22:29:18 2022] GET /identity/v3/users/neutron => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:18.415210 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bca51e-2542-4fec-a74c-483765a66e89 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:18.472763 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 322/662] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:29:18.472763 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a43df7-d013-4ad9-92e2-b98f779155ff None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:18.550426 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bca51e-2542-4fec-a74c-483765a66e89 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:18.589669 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da91cf-d417-4e2b-8f38-aa505a86fcfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:18.590410 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da91cf-d417-4e2b-8f38-aa505a86fcfc None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:18.590769 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da91cf-d417-4e2b-8f38-aa505a86fcfc None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:18.591118 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da91cf-d417-4e2b-8f38-aa505a86fcfc None admin] PATH_INFO: `/v3/projects/service` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:18.594396 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:18.599845 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5da91cf-d417-4e2b-8f38-aa505a86fcfc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:18.601437 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5da91cf-d417-4e2b-8f38-aa505a86fcfc None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:18.614607 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-e5da91cf-d417-4e2b-8f38-aa505a86fcfc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:18.614607 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:18.614607 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:18.614607 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:18.614607 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:18.614607 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:18.614607 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:18.614607 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:18.614607 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:18.614607 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:18.614607 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:18.614607 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:18.614607 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:29:18.614607 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:29:18.614607 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:29:18.615848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:29:18.615848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:18.615848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:18.615848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:18.615848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:18.615848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:18.615848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:18.615848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:18.615848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:18.615848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:18.615848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:18.615848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:18.615848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:18.615848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:18.615848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:18.615848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:18.615848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:18.617027 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:29:18.617027 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:29:18.617027 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:29:18.617027 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:29:18.617027 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:29:18.617027 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:29:18.617027 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:18.617027 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:18.623100 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 341/663] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:29:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:18.623100 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da91cf-d417-4e2b-8f38-aa505a86fcfc None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:18.624684 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:18.634222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a43df7-d013-4ad9-92e2-b98f779155ff None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:18.683038 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59248d02-0eb1-4aed-a46d-e404cd0af630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:18.683918 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59248d02-0eb1-4aed-a46d-e404cd0af630 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:18.684355 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59248d02-0eb1-4aed-a46d-e404cd0af630 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:18.684814 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59248d02-0eb1-4aed-a46d-e404cd0af630 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:18.760526 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da91cf-d417-4e2b-8f38-aa505a86fcfc None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:18.770443 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59248d02-0eb1-4aed-a46d-e404cd0af630 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:18.772098 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59248d02-0eb1-4aed-a46d-e404cd0af630 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:18.801282 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3111f74c-b514-44e4-9a79-a4f0b73d335a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:18.801875 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111f74c-b514-44e4-9a79-a4f0b73d335a None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:18.802168 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111f74c-b514-44e4-9a79-a4f0b73d335a None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:18.802435 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111f74c-b514-44e4-9a79-a4f0b73d335a None admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:18.803162 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3111f74c-b514-44e4-9a79-a4f0b73d335a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:18.803468 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3111f74c-b514-44e4-9a79-a4f0b73d335a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:18.804937 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3111f74c-b514-44e4-9a79-a4f0b73d335a None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:18.822784 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 342/664] 10.222.0.52 () {56 vars in 1139 bytes} [Mon Aug 29 22:29:18 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:18.823586 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3111f74c-b514-44e4-9a79-a4f0b73d335a None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:18.885864 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:18.890780 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:18.900756 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 323/665] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:29:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:29:18.900756 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59248d02-0eb1-4aed-a46d-e404cd0af630 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:18.939486 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3111f74c-b514-44e4-9a79-a4f0b73d335a None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:18.976067 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e80481e-9962-4c7e-82e3-49b8f3fd0f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:18.976680 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e80481e-9962-4c7e-82e3-49b8f3fd0f65 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:18.976992 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e80481e-9962-4c7e-82e3-49b8f3fd0f65 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:18.977267 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e80481e-9962-4c7e-82e3-49b8f3fd0f65 None admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:18.977958 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e80481e-9962-4c7e-82e3-49b8f3fd0f65 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:18.978265 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e80481e-9962-4c7e-82e3-49b8f3fd0f65 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:18.979756 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e80481e-9962-4c7e-82e3-49b8f3fd0f65 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:18.992615 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 343/666] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Aug 29 22:29:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:18.993494 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e80481e-9962-4c7e-82e3-49b8f3fd0f65 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:19.014118 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59248d02-0eb1-4aed-a46d-e404cd0af630 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:19.060628 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b95cc6-8847-472b-8602-85b1f4e389cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:19.061423 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b95cc6-8847-472b-8602-85b1f4e389cf None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:19.061700 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b95cc6-8847-472b-8602-85b1f4e389cf None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:19.061969 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b95cc6-8847-472b-8602-85b1f4e389cf None admin] PATH_INFO: `/v3/roles/service` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:19.073212 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b95cc6-8847-472b-8602-85b1f4e389cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:29:19.078168 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-71b95cc6-8847-472b-8602-85b1f4e389cf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:29:19.078168 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:19.078168 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:19.078168 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:19.078168 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:19.078168 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:19.078168 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:19.078168 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:19.078168 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:19.078168 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:19.078168 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:19.078168 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:19.078168 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:29:19.078168 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:29:19.078168 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:29:19.078168 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:29:19.079288 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:29:19.079288 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:29:19.079288 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:29:19.079288 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:29:19.079288 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:19.079288 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:19.079288 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:19.079288 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:19.079288 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:19.079288 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:19.079288 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:19.079288 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:19.079288 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:19.079288 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:19.079288 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:19.079288 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:19.081293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:19.081293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:19.081293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:19.081293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:19.081293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:29:19.081293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:29:19.081293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:29:19.081293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:29:19.081293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:29:19.081293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:29:19.081293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:29:19.081293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:19.081293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 324/667] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Aug 29 22:29:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:19.081293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b95cc6-8847-472b-8602-85b1f4e389cf None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:19.114643 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e80481e-9962-4c7e-82e3-49b8f3fd0f65 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:19.133681 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:19.141541 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:19.157513 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f37ce-3a6f-40a4-82b5-af881c0d8d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:19.158171 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f37ce-3a6f-40a4-82b5-af881c0d8d8b None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:19.158460 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f37ce-3a6f-40a4-82b5-af881c0d8d8b None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:19.158734 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f37ce-3a6f-40a4-82b5-af881c0d8d8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:19.188231 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b95cc6-8847-472b-8602-85b1f4e389cf None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:19.226682 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11f8ca-767b-434c-bf2c-4ae8b15c616b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:19.227467 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11f8ca-767b-434c-bf2c-4ae8b15c616b None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:19.227749 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11f8ca-767b-434c-bf2c-4ae8b15c616b None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:19.228023 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11f8ca-767b-434c-bf2c-4ae8b15c616b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:19.237101 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636f37ce-3a6f-40a4-82b5-af881c0d8d8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:19.238512 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636f37ce-3a6f-40a4-82b5-af881c0d8d8b None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:19.239841 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a11f8ca-767b-434c-bf2c-4ae8b15c616b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:19.241387 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a11f8ca-767b-434c-bf2c-4ae8b15c616b None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:19.257527 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-0a11f8ca-767b-434c-bf2c-4ae8b15c616b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:19.257527 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:19.257527 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:19.257527 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:19.257527 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:19.257527 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:19.257527 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:19.257527 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:19.257527 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:19.257527 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:19.257527 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:19.257527 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:19.257527 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:19.257527 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:19.257527 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:19.257527 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:19.258548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:19.258548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:19.258548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:19.258548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:19.258548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:19.258548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:19.258548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:19.258548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:19.258548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:19.258548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:19.258548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:19.258548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:19.258548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:19.258548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:19.258548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:19.258548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:19.258548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:19.259456 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:19.259456 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:19.259456 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:19.259456 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 325/668] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:19.259456 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11f8ca-767b-434c-bf2c-4ae8b15c616b None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:19.356228 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 344/669] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:29:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:29:19.357582 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f37ce-3a6f-40a4-82b5-af881c0d8d8b None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:19.371903 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11f8ca-767b-434c-bf2c-4ae8b15c616b None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:19.373173 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:19.386003 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:19.414209 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0576a1ae-6362-494e-99af-76edca9b3b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:19.414935 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0576a1ae-6362-494e-99af-76edca9b3b23 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:19.415218 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0576a1ae-6362-494e-99af-76edca9b3b23 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:19.415491 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0576a1ae-6362-494e-99af-76edca9b3b23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:19.416219 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0576a1ae-6362-494e-99af-76edca9b3b23 None admin] RBAC: Adding query filter params (role.id=db83c2f8e21945c3ab9642c620d29a23, scope.project.id=ee84c3340f2641ad963655a7a212ef96, user.id=6e782c8acaaa42c4bd78ddc7c8cc654b) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:19.416528 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0576a1ae-6362-494e-99af-76edca9b3b23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:19.418175 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0576a1ae-6362-494e-99af-76edca9b3b23 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:19.432751 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 326/670] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:29:19 2022] GET /identity/v3/role_assignments?user.id=6e782c8acaaa42c4bd78ddc7c8cc654b&scope.project.id=ee84c3340f2641ad963655a7a212ef96&role.id=db83c2f8e21945c3ab9642c620d29a23 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:19.432751 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0576a1ae-6362-494e-99af-76edca9b3b23 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:19.480408 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f37ce-3a6f-40a4-82b5-af881c0d8d8b None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:19.531473 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df40a03b-6eb1-428e-aea4-8ac6d4c7418e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:19.532098 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df40a03b-6eb1-428e-aea4-8ac6d4c7418e None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:19.532381 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df40a03b-6eb1-428e-aea4-8ac6d4c7418e None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:19.536220 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df40a03b-6eb1-428e-aea4-8ac6d4c7418e None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:19.537045 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df40a03b-6eb1-428e-aea4-8ac6d4c7418e None admin] RBAC: Adding query filter params (name=service) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:19.537428 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df40a03b-6eb1-428e-aea4-8ac6d4c7418e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:19.543070 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df40a03b-6eb1-428e-aea4-8ac6d4c7418e None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:19.556314 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 345/671] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Aug 29 22:29:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:19.557347 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df40a03b-6eb1-428e-aea4-8ac6d4c7418e None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:19.574880 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0576a1ae-6362-494e-99af-76edca9b3b23 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:19.618563 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089742f7-864e-42b0-9e0d-63dab9a57c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:19.619149 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089742f7-864e-42b0-9e0d-63dab9a57c7f None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:19.619421 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089742f7-864e-42b0-9e0d-63dab9a57c7f None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:19.619689 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089742f7-864e-42b0-9e0d-63dab9a57c7f None admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:19.620334 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089742f7-864e-42b0-9e0d-63dab9a57c7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:19.620633 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089742f7-864e-42b0-9e0d-63dab9a57c7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:19.622173 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089742f7-864e-42b0-9e0d-63dab9a57c7f None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:19.635901 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 327/672] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:19.636576 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089742f7-864e-42b0-9e0d-63dab9a57c7f None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:19.668657 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:19.679351 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:19.699598 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df40a03b-6eb1-428e-aea4-8ac6d4c7418e None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:19.736465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2280a5-35b3-41c7-9ebf-49707fc8ce7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:19.737109 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2280a5-35b3-41c7-9ebf-49707fc8ce7c None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:19.737369 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2280a5-35b3-41c7-9ebf-49707fc8ce7c None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:19.737642 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2280a5-35b3-41c7-9ebf-49707fc8ce7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:19.749723 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2280a5-35b3-41c7-9ebf-49707fc8ce7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:19.751167 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2280a5-35b3-41c7-9ebf-49707fc8ce7c None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:19.761927 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089742f7-864e-42b0-9e0d-63dab9a57c7f None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:19.766099 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-ca2280a5-35b3-41c7-9ebf-49707fc8ce7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:19.766099 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:19.766099 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:19.766099 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:19.766099 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:19.766099 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:19.766099 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:19.766099 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:19.766099 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:19.766099 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:19.766099 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:19.766099 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:19.766099 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:19.766099 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:19.766099 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:19.766099 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:19.767138 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:19.767138 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:19.767138 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:19.767138 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:19.767138 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:19.767138 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:19.767138 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:19.767138 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:19.767138 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:19.767138 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:19.767138 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:19.767138 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:19.767138 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:19.767138 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:19.767138 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:19.767138 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:19.767138 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:19.768845 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:19.768845 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:19.768845 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:19.768845 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 346/673] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:19.769532 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2280a5-35b3-41c7-9ebf-49707fc8ce7c None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:19.816071 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4822677-6c7f-4515-b350-a350d210546b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:19.816976 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4822677-6c7f-4515-b350-a350d210546b None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:19.817405 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4822677-6c7f-4515-b350-a350d210546b None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:19.817858 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4822677-6c7f-4515-b350-a350d210546b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:19.829115 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4822677-6c7f-4515-b350-a350d210546b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:19.831609 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4822677-6c7f-4515-b350-a350d210546b None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:19.848174 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-c4822677-6c7f-4515-b350-a350d210546b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:19.848174 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:19.848174 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:19.848174 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:19.848174 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:19.848174 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:19.848174 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:19.848174 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:19.848174 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:19.848174 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:19.848174 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:19.848174 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:19.848174 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:19.848174 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:19.848174 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:19.848174 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:19.849230 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:19.849230 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:19.849230 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:19.849230 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:19.849230 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:19.849230 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:19.849230 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:19.849230 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:19.849230 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:19.849230 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:19.849230 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:19.849230 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:19.849230 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:19.849230 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:19.849230 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:19.849230 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:19.849230 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:19.850120 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:19.850120 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:19.850120 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:19.851931 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 328/674] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:19.853013 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4822677-6c7f-4515-b350-a350d210546b None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:19.896974 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2280a5-35b3-41c7-9ebf-49707fc8ce7c None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:19.925976 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:19.934780 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52330667-d083-41e3-b53e-e6659c21340a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:19.940848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52330667-d083-41e3-b53e-e6659c21340a None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:19.940848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52330667-d083-41e3-b53e-e6659c21340a None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:19.940848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52330667-d083-41e3-b53e-e6659c21340a None admin] PATH_INFO: `/v3/projects/service` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:19.944178 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52330667-d083-41e3-b53e-e6659c21340a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:19.945801 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52330667-d083-41e3-b53e-e6659c21340a None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:19.949275 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:19.960092 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-52330667-d083-41e3-b53e-e6659c21340a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:19.960092 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:19.960092 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:19.960092 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:19.960092 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:19.960092 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:19.960092 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:19.960092 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:19.960092 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:19.960092 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:19.960092 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:19.960092 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:19.960092 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:29:19.960092 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:29:19.960092 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:29:19.961082 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:29:19.961082 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:19.961082 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:19.961082 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:19.961082 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:19.961082 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:19.961082 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:19.961082 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:19.961082 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:19.961082 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:19.961082 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:19.961082 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:19.961082 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:19.961082 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:19.961082 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:19.961082 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:19.961082 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:19.964623 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:29:19.964623 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:29:19.964623 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:29:19.964623 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:29:19.964623 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:29:19.964623 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:29:19.964623 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:19.964623 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:19.964623 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 347/675] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:29:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:19.964623 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52330667-d083-41e3-b53e-e6659c21340a None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:19.970024 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4822677-6c7f-4515-b350-a350d210546b None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:20.011508 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39801b63-82a5-42be-82cd-8d7a6ff52af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:20.012370 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39801b63-82a5-42be-82cd-8d7a6ff52af2 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:20.012843 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39801b63-82a5-42be-82cd-8d7a6ff52af2 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:20.013266 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39801b63-82a5-42be-82cd-8d7a6ff52af2 None admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:20.014057 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39801b63-82a5-42be-82cd-8d7a6ff52af2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:20.014506 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39801b63-82a5-42be-82cd-8d7a6ff52af2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:20.016136 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39801b63-82a5-42be-82cd-8d7a6ff52af2 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:20.033024 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 329/676] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:20.034944 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39801b63-82a5-42be-82cd-8d7a6ff52af2 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:20.081281 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52330667-d083-41e3-b53e-e6659c21340a None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:20.120397 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc23951-76c9-4f7b-9146-9adcff259a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:20.121028 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc23951-76c9-4f7b-9146-9adcff259a8a None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:20.121307 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc23951-76c9-4f7b-9146-9adcff259a8a None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:20.121575 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc23951-76c9-4f7b-9146-9adcff259a8a None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:20.122252 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc23951-76c9-4f7b-9146-9adcff259a8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:20.122554 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc23951-76c9-4f7b-9146-9adcff259a8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:20.124112 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc23951-76c9-4f7b-9146-9adcff259a8a None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:20.141225 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 348/677] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:20.141225 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc23951-76c9-4f7b-9146-9adcff259a8a None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:20.156821 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39801b63-82a5-42be-82cd-8d7a6ff52af2 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:20.194805 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962b32c-296f-4d8e-89f8-71a7d8aaa8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:20.195418 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962b32c-296f-4d8e-89f8-71a7d8aaa8ae None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:20.195704 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962b32c-296f-4d8e-89f8-71a7d8aaa8ae None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:20.195986 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962b32c-296f-4d8e-89f8-71a7d8aaa8ae None admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:20.196683 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1962b32c-296f-4d8e-89f8-71a7d8aaa8ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:20.197021 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1962b32c-296f-4d8e-89f8-71a7d8aaa8ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:20.198459 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1962b32c-296f-4d8e-89f8-71a7d8aaa8ae None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:20.199865 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:20.211190 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 330/678] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Aug 29 22:29:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:20.212418 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c200b-5e9c-4a59-9193-fa0d7ff31f5a None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:20.212708 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c200b-5e9c-4a59-9193-fa0d7ff31f5a None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:20.216827 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c200b-5e9c-4a59-9193-fa0d7ff31f5a None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:20.216827 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 331/679] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:29:20.216827 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c200b-5e9c-4a59-9193-fa0d7ff31f5a None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:20.227971 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:20.264883 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc23951-76c9-4f7b-9146-9adcff259a8a None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:20.301843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6191a5b3-e929-4d25-84e2-90c5a2f566ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:20.302470 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191a5b3-e929-4d25-84e2-90c5a2f566ef None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:20.302751 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191a5b3-e929-4d25-84e2-90c5a2f566ef None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:20.303025 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191a5b3-e929-4d25-84e2-90c5a2f566ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:20.309851 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6191a5b3-e929-4d25-84e2-90c5a2f566ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:20.311291 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6191a5b3-e929-4d25-84e2-90c5a2f566ef None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:20.323287 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-6191a5b3-e929-4d25-84e2-90c5a2f566ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:20.323287 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:20.323287 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:20.323287 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:20.323287 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:20.323287 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:20.323287 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:20.323287 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:20.323287 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:20.323287 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:20.323287 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:20.323287 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:20.323287 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:29:20.323287 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:29:20.323287 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:29:20.324384 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:29:20.324384 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:20.324384 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:20.324384 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:20.324384 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:20.324384 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:20.324384 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:20.324384 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:20.324384 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:20.324384 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:20.324384 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:20.324384 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:20.324384 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:20.324384 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:20.324384 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:20.324384 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:20.324384 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:20.325333 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:29:20.325333 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:29:20.325333 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:29:20.325333 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:29:20.325333 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:29:20.325333 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:29:20.325333 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:20.325333 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:20.325333 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 349/680] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:29:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:20.325333 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6191a5b3-e929-4d25-84e2-90c5a2f566ef None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:20.329711 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c200b-5e9c-4a59-9193-fa0d7ff31f5a None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:20.368878 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045889ca-1308-4173-b9df-df2219229b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:20.369768 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045889ca-1308-4173-b9df-df2219229b3b None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:20.370284 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045889ca-1308-4173-b9df-df2219229b3b None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:20.370704 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045889ca-1308-4173-b9df-df2219229b3b None admin] PATH_INFO: `/v3/users/swift` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:20.378985 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045889ca-1308-4173-b9df-df2219229b3b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:20.380435 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045889ca-1308-4173-b9df-df2219229b3b None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:20.392321 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-045889ca-1308-4173-b9df-df2219229b3b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 22:29:20.392321 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:20.392321 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:20.392321 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:20.392321 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:20.392321 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:20.392321 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:20.392321 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:20.392321 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:20.392321 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:20.392321 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:20.392321 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:20.392321 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:29:20.392321 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:29:20.392321 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:29:20.392321 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:29:20.394064 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:20.394064 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:20.394064 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:29:20.394064 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:20.394064 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:29:20.394064 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:20.394064 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:20.394064 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:20.394064 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:20.394064 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:20.394064 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:20.394064 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:20.394064 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:20.394064 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:20.394064 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:20.394064 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:20.394064 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:20.394064 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:20.395102 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:20.395102 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:20.395102 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:29:20.395102 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:29:20.395102 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:29:20.395102 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:29:20.395102 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:29:20.395102 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:29:20.395102 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 22:29:20.395102 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:20.398907 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 332/681] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Aug 29 22:29:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:20.400179 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07542280-7e13-465c-84e5-6276dddc2f3b None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:20.400444 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07542280-7e13-465c-84e5-6276dddc2f3b None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:20.400741 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07542280-7e13-465c-84e5-6276dddc2f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:20.424690 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-07542280-7e13-465c-84e5-6276dddc2f3b None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:20.427127 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-07542280-7e13-465c-84e5-6276dddc2f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:20.438400 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6191a5b3-e929-4d25-84e2-90c5a2f566ef None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:20.447928 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:20.477240 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dfef33-477c-4f4d-bbc9-708e972d108d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:20.477852 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dfef33-477c-4f4d-bbc9-708e972d108d None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:20.478491 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dfef33-477c-4f4d-bbc9-708e972d108d None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:20.478776 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dfef33-477c-4f4d-bbc9-708e972d108d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:20.479863 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dfef33-477c-4f4d-bbc9-708e972d108d None admin] RBAC: Adding query filter params (role.id=db83c2f8e21945c3ab9642c620d29a23, scope.project.id=ee84c3340f2641ad963655a7a212ef96, user.id=bdbf961e02934e88af3a72d6218f32fa) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:20.480177 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dfef33-477c-4f4d-bbc9-708e972d108d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:20.482241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dfef33-477c-4f4d-bbc9-708e972d108d None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:20.500749 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 350/682] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:29:20 2022] GET /identity/v3/role_assignments?user.id=bdbf961e02934e88af3a72d6218f32fa&scope.project.id=ee84c3340f2641ad963655a7a212ef96&role.id=db83c2f8e21945c3ab9642c620d29a23 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:20.500749 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dfef33-477c-4f4d-bbc9-708e972d108d None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:20.501058 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:20.510027 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 333/683] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:29:20.515155 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07542280-7e13-465c-84e5-6276dddc2f3b None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:20.700461 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07542280-7e13-465c-84e5-6276dddc2f3b None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:20.718544 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dfef33-477c-4f4d-bbc9-708e972d108d None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:20.740162 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847a19bb-4131-451f-8052-b58c537cce03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:20.740801 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a19bb-4131-451f-8052-b58c537cce03 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:20.741086 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a19bb-4131-451f-8052-b58c537cce03 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:20.741364 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a19bb-4131-451f-8052-b58c537cce03 None admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:20.742105 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847a19bb-4131-451f-8052-b58c537cce03 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:20.742409 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847a19bb-4131-451f-8052-b58c537cce03 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:20.744701 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847a19bb-4131-451f-8052-b58c537cce03 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:20.760179 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d78d8b-fc8c-4ec6-ad41-c920abd99df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:20.760817 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d78d8b-fc8c-4ec6-ad41-c920abd99df4 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:20.761101 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d78d8b-fc8c-4ec6-ad41-c920abd99df4 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:20.761377 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d78d8b-fc8c-4ec6-ad41-c920abd99df4 None admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:20.762551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d78d8b-fc8c-4ec6-ad41-c920abd99df4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:20.762894 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d78d8b-fc8c-4ec6-ad41-c920abd99df4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:20.764783 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d78d8b-fc8c-4ec6-ad41-c920abd99df4 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:20.767182 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 334/684] 10.222.0.52 () {56 vars in 1135 bytes} [Mon Aug 29 22:29:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:20.773990 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847a19bb-4131-451f-8052-b58c537cce03 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:20.781825 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 351/685] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Aug 29 22:29:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:20.789243 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d78d8b-fc8c-4ec6-ad41-c920abd99df4 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:20.829378 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:20.844619 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:20.906790 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847a19bb-4131-451f-8052-b58c537cce03 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:20.921071 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d78d8b-fc8c-4ec6-ad41-c920abd99df4 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:20.957640 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32dc5e0-5e90-4698-ac0c-b9d92af959c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:20.959592 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32dc5e0-5e90-4698-ac0c-b9d92af959c6 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:20.960049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32dc5e0-5e90-4698-ac0c-b9d92af959c6 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:20.960464 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32dc5e0-5e90-4698-ac0c-b9d92af959c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:20.966651 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9249c33f-4a0c-4f0c-b7c7-853f02d4a1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:20.967431 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9249c33f-4a0c-4f0c-b7c7-853f02d4a1a2 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:20.967810 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9249c33f-4a0c-4f0c-b7c7-853f02d4a1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:20.968184 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9249c33f-4a0c-4f0c-b7c7-853f02d4a1a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:20.969048 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9249c33f-4a0c-4f0c-b7c7-853f02d4a1a2 None admin] RBAC: Adding query filter params (role.id=db83c2f8e21945c3ab9642c620d29a23, scope.project.id=ee84c3340f2641ad963655a7a212ef96, user.id=0fe5e8c4cb324ffb9d5b80f18d123727) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:20.969508 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9249c33f-4a0c-4f0c-b7c7-853f02d4a1a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:20.971099 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9249c33f-4a0c-4f0c-b7c7-853f02d4a1a2 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:21.028706 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 352/686] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:29:20 2022] GET /identity/v3/role_assignments?user.id=0fe5e8c4cb324ffb9d5b80f18d123727&scope.project.id=ee84c3340f2641ad963655a7a212ef96&role.id=db83c2f8e21945c3ab9642c620d29a23 => generated 608 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:29:21.029924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9249c33f-4a0c-4f0c-b7c7-853f02d4a1a2 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:21.040411 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32dc5e0-5e90-4698-ac0c-b9d92af959c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:21.042155 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32dc5e0-5e90-4698-ac0c-b9d92af959c6 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:21.170381 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:21.173179 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:21.179412 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9249c33f-4a0c-4f0c-b7c7-853f02d4a1a2 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:21.224506 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def6449-1648-4d68-8e45-eaddb660a246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:21.225587 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def6449-1648-4d68-8e45-eaddb660a246 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:21.226174 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def6449-1648-4d68-8e45-eaddb660a246 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:21.226651 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def6449-1648-4d68-8e45-eaddb660a246 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:21.240152 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1def6449-1648-4d68-8e45-eaddb660a246 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:21.242188 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1def6449-1648-4d68-8e45-eaddb660a246 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:21.259519 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-1def6449-1648-4d68-8e45-eaddb660a246 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:21.259519 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:21.259519 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:21.259519 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:21.259519 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:21.259519 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:21.259519 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:21.259519 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:21.259519 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:21.259519 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:21.259519 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:21.259519 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:21.259519 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:21.259519 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:21.259519 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:21.259519 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:21.260551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:21.260551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:21.260551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:21.260551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:21.260551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:21.260551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:21.260551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:21.260551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:21.260551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:21.260551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:21.260551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:21.260551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:21.260551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:21.260551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:21.260551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:21.260551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:21.260551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:21.266662 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:21.266662 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:21.266662 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:21.266850 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 335/687] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:29:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:29:21.269607 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 353/688] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:21.271234 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32dc5e0-5e90-4698-ac0c-b9d92af959c6 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:21.274815 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def6449-1648-4d68-8e45-eaddb660a246 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:21.393650 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def6449-1648-4d68-8e45-eaddb660a246 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:21.401958 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32dc5e0-5e90-4698-ac0c-b9d92af959c6 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:21.433236 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2b92d6-e9e1-475c-8963-f6b7cea5bf84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:21.434257 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b92d6-e9e1-475c-8963-f6b7cea5bf84 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:21.434774 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b92d6-e9e1-475c-8963-f6b7cea5bf84 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:21.435244 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b92d6-e9e1-475c-8963-f6b7cea5bf84 None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:21.436208 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2b92d6-e9e1-475c-8963-f6b7cea5bf84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:21.436795 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2b92d6-e9e1-475c-8963-f6b7cea5bf84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:21.438446 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2b92d6-e9e1-475c-8963-f6b7cea5bf84 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:21.442301 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adebaf2-4b0b-41ed-bab8-1bae8bbdede3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:21.442945 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adebaf2-4b0b-41ed-bab8-1bae8bbdede3 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:21.443227 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adebaf2-4b0b-41ed-bab8-1bae8bbdede3 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:21.443503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adebaf2-4b0b-41ed-bab8-1bae8bbdede3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:21.453903 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 354/689] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:29:21.457887 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adebaf2-4b0b-41ed-bab8-1bae8bbdede3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:21.460304 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adebaf2-4b0b-41ed-bab8-1bae8bbdede3 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:21.461469 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2b92d6-e9e1-475c-8963-f6b7cea5bf84 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:21.475699 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:21.484603 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-7adebaf2-4b0b-41ed-bab8-1bae8bbdede3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:21.484603 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:21.484603 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:21.484603 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:21.484603 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:21.484603 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:21.484603 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:21.484603 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:21.484603 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:21.484603 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:21.484603 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:21.484603 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:21.484603 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:21.484603 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:21.484603 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:21.484603 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:21.493852 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:21.493852 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:21.493852 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:21.493852 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:21.493852 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:21.493852 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:21.493852 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:21.493852 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:21.493852 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:21.493852 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:21.493852 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:21.493852 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:21.493852 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:21.493852 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:21.493852 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:21.493852 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:21.493852 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:21.494830 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:21.494830 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:21.494830 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:21.494830 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 336/690] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:21.498500 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adebaf2-4b0b-41ed-bab8-1bae8bbdede3 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:21.513308 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:21.595240 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2b92d6-e9e1-475c-8963-f6b7cea5bf84 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:21.640264 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adebaf2-4b0b-41ed-bab8-1bae8bbdede3 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:21.648224 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b447fc-78b5-44af-998c-d0b0c7fa2862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:21.649129 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b447fc-78b5-44af-998c-d0b0c7fa2862 None admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:21.650181 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b447fc-78b5-44af-998c-d0b0c7fa2862 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:21.650651 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b447fc-78b5-44af-998c-d0b0c7fa2862 None admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:21.651435 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b447fc-78b5-44af-998c-d0b0c7fa2862 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:21.653930 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b447fc-78b5-44af-998c-d0b0c7fa2862 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:21.684479 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334437a8-3869-4adc-94a7-49342bede034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:21.685408 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334437a8-3869-4adc-94a7-49342bede034 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:21.688179 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334437a8-3869-4adc-94a7-49342bede034 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:21.688179 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334437a8-3869-4adc-94a7-49342bede034 None admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:21.688179 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334437a8-3869-4adc-94a7-49342bede034 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:21.688179 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334437a8-3869-4adc-94a7-49342bede034 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:21.688968 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334437a8-3869-4adc-94a7-49342bede034 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:21.708543 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 337/691] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:21.708543 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e29b0-8b7e-434a-ade9-2eec864699cd None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:21.708543 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e29b0-8b7e-434a-ade9-2eec864699cd None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:21.708543 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e29b0-8b7e-434a-ade9-2eec864699cd None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:21.708543 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 338/692] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:29:21.713605 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e29b0-8b7e-434a-ade9-2eec864699cd None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:21.760599 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 355/693] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Aug 29 22:29:21 2022] POST /identity/v3/users => generated 254 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:29:21.762148 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abcd56-8fb7-4fa3-8431-e7cec7d1c2ff None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:21.762462 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abcd56-8fb7-4fa3-8431-e7cec7d1c2ff None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:21.762695 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abcd56-8fb7-4fa3-8431-e7cec7d1c2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:21.792812 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-20abcd56-8fb7-4fa3-8431-e7cec7d1c2ff None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:21.792812 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-20abcd56-8fb7-4fa3-8431-e7cec7d1c2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:21.824765 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:21.877266 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:21.894777 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 356/694] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:29:21.905463 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e29b0-8b7e-434a-ade9-2eec864699cd None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:21.957161 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55fbb8-643c-43aa-bb94-b55924ebd483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:21.957805 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55fbb8-643c-43aa-bb94-b55924ebd483 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:21.958349 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55fbb8-643c-43aa-bb94-b55924ebd483 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:21.958456 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55fbb8-643c-43aa-bb94-b55924ebd483 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:21.965857 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c55fbb8-643c-43aa-bb94-b55924ebd483 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:21.967461 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c55fbb8-643c-43aa-bb94-b55924ebd483 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:21.981232 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-6c55fbb8-643c-43aa-bb94-b55924ebd483 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:21.981232 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:21.981232 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:21.981232 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:21.981232 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:21.981232 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:21.981232 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:21.981232 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:21.981232 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:21.981232 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:21.981232 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:21.981232 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:21.981232 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:29:21.981232 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:29:21.981232 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:29:21.982141 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:29:21.982141 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:21.982141 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:21.982141 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:21.982141 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:21.982141 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:21.982141 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:21.982141 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:21.982141 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:21.982141 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:21.982141 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:21.982141 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:21.982141 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:21.982141 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:21.982141 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:21.982141 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:21.982141 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:21.983020 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:29:21.983020 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:29:21.983020 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:29:21.983020 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:29:21.983020 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:29:21.983020 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:29:21.983020 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:21.983020 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:21.987915 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 339/695] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:29:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:21.989308 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20abcd56-8fb7-4fa3-8431-e7cec7d1c2ff None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:22.101368 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20abcd56-8fb7-4fa3-8431-e7cec7d1c2ff None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:22.138222 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138969a6-ab25-4b54-9c85-68c9e4718e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:22.138834 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138969a6-ab25-4b54-9c85-68c9e4718e63 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:22.139116 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138969a6-ab25-4b54-9c85-68c9e4718e63 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:22.139391 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138969a6-ab25-4b54-9c85-68c9e4718e63 None admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:22.140078 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138969a6-ab25-4b54-9c85-68c9e4718e63 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:22.140382 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138969a6-ab25-4b54-9c85-68c9e4718e63 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:22.141918 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138969a6-ab25-4b54-9c85-68c9e4718e63 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:22.160765 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 357/696] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Aug 29 22:29:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:22.162204 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55fbb8-643c-43aa-bb94-b55924ebd483 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:22.207925 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138969a6-ab25-4b54-9c85-68c9e4718e63 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:22.272361 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:22.286072 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55fbb8-643c-43aa-bb94-b55924ebd483 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:22.321256 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:22.331844 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00809430-a20e-48fa-9b44-377c8c9ae0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:22.332791 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00809430-a20e-48fa-9b44-377c8c9ae0a9 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:22.334175 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00809430-a20e-48fa-9b44-377c8c9ae0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:22.334175 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00809430-a20e-48fa-9b44-377c8c9ae0a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:22.334546 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00809430-a20e-48fa-9b44-377c8c9ae0a9 None admin] RBAC: Adding query filter params (role.id=db83c2f8e21945c3ab9642c620d29a23, scope.project.id=ee84c3340f2641ad963655a7a212ef96, user.id=57c175ac90e545358746176972ff39a3) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:22.334993 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00809430-a20e-48fa-9b44-377c8c9ae0a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:22.336907 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00809430-a20e-48fa-9b44-377c8c9ae0a9 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:22.343948 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 340/697] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:29:22 2022] GET /identity/v3/role_assignments?user.id=57c175ac90e545358746176972ff39a3&scope.project.id=ee84c3340f2641ad963655a7a212ef96&role.id=db83c2f8e21945c3ab9642c620d29a23 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:22.346906 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138969a6-ab25-4b54-9c85-68c9e4718e63 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:22.389211 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d2c01d-745c-4dd1-b2b0-9201f19d4c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:22.390108 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2c01d-745c-4dd1-b2b0-9201f19d4c72 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:22.390557 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2c01d-745c-4dd1-b2b0-9201f19d4c72 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:22.390971 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2c01d-745c-4dd1-b2b0-9201f19d4c72 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:22.391700 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d2c01d-745c-4dd1-b2b0-9201f19d4c72 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:22.393558 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d2c01d-745c-4dd1-b2b0-9201f19d4c72 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:22.406494 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-15d2c01d-745c-4dd1-b2b0-9201f19d4c72 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 22:29:22.406494 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:22.406494 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:29:22.406494 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:29:22.406494 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:29:22.406494 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:29:22.406494 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:29:22.406494 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:29:22.406494 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 22:29:22.406494 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:22.406494 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:29:22.406494 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:22.406494 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:22.406494 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:22.406494 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:22.406494 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:22.406494 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:22.407804 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:22.407804 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:22.407804 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:22.407804 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:22.407804 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:22.407804 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:22.407804 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:29:22.407804 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:29:22.407804 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:29:22.407804 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:29:22.407804 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:22.407804 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:22.407804 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:22.407804 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:22.407804 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:22.407804 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:22.408955 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:22.408955 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:22.408955 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:22.408955 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:22.408955 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:22.408955 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:22.408955 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:22.408955 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:22.408955 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:22.408955 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:22.408955 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:29:22.408955 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:29:22.408955 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 22:29:22.408955 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:22.411542 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 358/698] 10.222.0.52 () {56 vars in 1129 bytes} [Mon Aug 29 22:29:22 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:29:22.417692 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00809430-a20e-48fa-9b44-377c8c9ae0a9 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:22.569798 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00809430-a20e-48fa-9b44-377c8c9ae0a9 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:22.612873 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba4560-bf2f-4a2d-8a9a-c5ad5614eb44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:22.613760 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba4560-bf2f-4a2d-8a9a-c5ad5614eb44 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:22.614185 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba4560-bf2f-4a2d-8a9a-c5ad5614eb44 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:22.614649 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba4560-bf2f-4a2d-8a9a-c5ad5614eb44 None admin] PATH_INFO: `/v3/services` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:22.615445 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba4560-bf2f-4a2d-8a9a-c5ad5614eb44 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:22.615890 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba4560-bf2f-4a2d-8a9a-c5ad5614eb44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:22.617494 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba4560-bf2f-4a2d-8a9a-c5ad5614eb44 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:22.623678 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 341/699] 10.222.0.52 () {56 vars in 1124 bytes} [Mon Aug 29 22:29:22 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:22.629597 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d2c01d-745c-4dd1-b2b0-9201f19d4c72 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:22.741048 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d2c01d-745c-4dd1-b2b0-9201f19d4c72 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:22.760656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:22.778812 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2666f4-ceaa-4a5c-a518-eacb23cad331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:22.779423 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2666f4-ceaa-4a5c-a518-eacb23cad331 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:22.779703 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2666f4-ceaa-4a5c-a518-eacb23cad331 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:22.779973 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2666f4-ceaa-4a5c-a518-eacb23cad331 None admin] PATH_INFO: `/v3/services` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:22.780730 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2666f4-ceaa-4a5c-a518-eacb23cad331 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:22.781043 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2666f4-ceaa-4a5c-a518-eacb23cad331 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:22.782453 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2666f4-ceaa-4a5c-a518-eacb23cad331 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:22.789413 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 359/700] 10.222.0.52 () {56 vars in 1124 bytes} [Mon Aug 29 22:29:22 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:22.902188 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924cfbd-11bd-46f7-a4ea-09e6ad371613 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:22.902527 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924cfbd-11bd-46f7-a4ea-09e6ad371613 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:22.902749 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924cfbd-11bd-46f7-a4ea-09e6ad371613 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:22.904471 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 342/701] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:29:22.910160 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981cd4d-9f86-41a5-ac9c-548d6e616f67 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:22.910450 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981cd4d-9f86-41a5-ac9c-548d6e616f67 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:22.910718 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981cd4d-9f86-41a5-ac9c-548d6e616f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:22.931120 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-3981cd4d-9f86-41a5-ac9c-548d6e616f67 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:22.932322 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-3981cd4d-9f86-41a5-ac9c-548d6e616f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:23.011834 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 360/702] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:29:23.412066 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924cfbd-11bd-46f7-a4ea-09e6ad371613 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:23.492351 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:23.556015 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924cfbd-11bd-46f7-a4ea-09e6ad371613 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:23.591257 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4d3071-cb97-4095-a2e7-8ce99a0476b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:23.591890 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d3071-cb97-4095-a2e7-8ce99a0476b0 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:23.592167 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d3071-cb97-4095-a2e7-8ce99a0476b0 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:23.592438 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d3071-cb97-4095-a2e7-8ce99a0476b0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:23.603598 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4d3071-cb97-4095-a2e7-8ce99a0476b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:29:23.609841 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-8e4d3071-cb97-4095-a2e7-8ce99a0476b0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:29:23.609841 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:23.609841 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:23.609841 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:23.609841 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:23.609841 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:23.609841 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:23.609841 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:23.609841 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:23.609841 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:23.609841 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:23.609841 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:23.609841 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:29:23.609841 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:29:23.609841 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:29:23.609841 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:29:23.610753 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:29:23.610753 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:29:23.610753 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:29:23.610753 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:29:23.610753 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:23.610753 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:23.610753 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:23.610753 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:23.610753 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:23.610753 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:23.610753 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:23.610753 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:23.610753 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:23.610753 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:23.610753 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:23.610753 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:23.612877 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:23.612877 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:23.612877 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:23.612877 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:23.612877 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:29:23.612877 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:29:23.612877 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:29:23.612877 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:29:23.612877 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:29:23.612877 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:29:23.612877 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:29:23.612877 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:23.612877 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 343/703] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Aug 29 22:29:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:23.617557 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3981cd4d-9f86-41a5-ac9c-548d6e616f67 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:23.762831 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3981cd4d-9f86-41a5-ac9c-548d6e616f67 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:23.778914 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:23.810432 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492d4cc9-635e-4436-84c9-46b691f29b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:23.811040 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d4cc9-635e-4436-84c9-46b691f29b70 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:23.811321 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d4cc9-635e-4436-84c9-46b691f29b70 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:23.811585 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d4cc9-635e-4436-84c9-46b691f29b70 None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:23.812245 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492d4cc9-635e-4436-84c9-46b691f29b70 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:23.812542 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492d4cc9-635e-4436-84c9-46b691f29b70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:23.813929 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492d4cc9-635e-4436-84c9-46b691f29b70 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:23.821673 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed8a9f-f1db-4b49-880b-6140c30288cc None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:23.821955 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed8a9f-f1db-4b49-880b-6140c30288cc None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:23.822235 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed8a9f-f1db-4b49-880b-6140c30288cc None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:23.824347 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 344/704] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:29:23.829376 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5850e9-fa34-4cf4-9d24-c0084bb8c34a None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:23.829644 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5850e9-fa34-4cf4-9d24-c0084bb8c34a None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:23.829920 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5850e9-fa34-4cf4-9d24-c0084bb8c34a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:23.835189 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 361/705] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:29:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:23.845082 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492d4cc9-635e-4436-84c9-46b691f29b70 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:23.859583 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-8d5850e9-fa34-4cf4-9d24-c0084bb8c34a None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:23.860801 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-8d5850e9-fa34-4cf4-9d24-c0084bb8c34a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:23.985966 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492d4cc9-635e-4436-84c9-46b691f29b70 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:24.012959 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 345/706] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:29:24.035777 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f26c5-c0b0-4b0d-a03b-ef6ee64269b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:24.036404 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f26c5-c0b0-4b0d-a03b-ef6ee64269b3 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:24.036684 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f26c5-c0b0-4b0d-a03b-ef6ee64269b3 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:24.036981 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f26c5-c0b0-4b0d-a03b-ef6ee64269b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:24.048032 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231f26c5-c0b0-4b0d-a03b-ef6ee64269b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:24.050596 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231f26c5-c0b0-4b0d-a03b-ef6ee64269b3 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:24.065673 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-231f26c5-c0b0-4b0d-a03b-ef6ee64269b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:24.065673 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:24.065673 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:24.065673 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:24.065673 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:24.065673 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:24.065673 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:24.065673 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:24.065673 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:24.065673 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:24.065673 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:24.065673 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:24.065673 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:24.065673 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:24.065673 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:24.065673 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:24.066698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:24.066698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:24.066698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:24.066698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:24.066698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:24.066698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:24.066698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:24.066698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:24.066698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:24.066698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:24.066698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:24.066698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:24.066698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:24.066698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:24.066698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:24.066698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:24.066698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:24.068227 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:24.068227 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:24.068227 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:24.068227 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 362/707] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:24.073001 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5850e9-fa34-4cf4-9d24-c0084bb8c34a None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:24.122841 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:24.179396 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5850e9-fa34-4cf4-9d24-c0084bb8c34a None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:24.215068 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72751271-7e5b-4777-8499-246d890b0d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:24.215680 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72751271-7e5b-4777-8499-246d890b0d15 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:24.215954 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72751271-7e5b-4777-8499-246d890b0d15 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:24.216222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72751271-7e5b-4777-8499-246d890b0d15 None admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:24.216895 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72751271-7e5b-4777-8499-246d890b0d15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:24.217196 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72751271-7e5b-4777-8499-246d890b0d15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:24.218630 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72751271-7e5b-4777-8499-246d890b0d15 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:24.230943 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 346/708] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:24.236995 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f26c5-c0b0-4b0d-a03b-ef6ee64269b3 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:24.270979 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:24.375452 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f26c5-c0b0-4b0d-a03b-ef6ee64269b3 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:24.388089 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72751271-7e5b-4777-8499-246d890b0d15 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:24.423949 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44c0b95-e5c8-437a-b38d-f8d84bf37879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:24.424867 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c0b95-e5c8-437a-b38d-f8d84bf37879 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:24.425152 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c0b95-e5c8-437a-b38d-f8d84bf37879 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:24.425427 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c0b95-e5c8-437a-b38d-f8d84bf37879 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:24.436359 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44c0b95-e5c8-437a-b38d-f8d84bf37879 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:24.437836 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44c0b95-e5c8-437a-b38d-f8d84bf37879 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:24.456959 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-d44c0b95-e5c8-437a-b38d-f8d84bf37879 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 22:29:24.456959 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:24.456959 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:24.456959 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:24.456959 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:24.456959 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:24.456959 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:24.456959 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:24.456959 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:24.456959 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:24.456959 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:24.456959 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:24.456959 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:29:24.456959 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:29:24.456959 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:29:24.456959 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:29:24.458022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:24.458022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:24.458022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:29:24.458022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:24.458022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:29:24.458022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:24.458022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:24.458022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:24.458022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:24.458022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:24.458022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:24.458022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:24.458022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:24.458022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:24.458022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:24.458022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:24.458022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:24.458022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:24.459376 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:24.459376 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:24.459376 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:29:24.459376 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:29:24.459376 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:29:24.459376 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:29:24.459376 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:29:24.459376 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:29:24.459376 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 22:29:24.459376 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:24.459376 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 363/709] 10.222.0.52 () {56 vars in 1093 bytes} [Mon Aug 29 22:29:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:24.465003 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44c0b95-e5c8-437a-b38d-f8d84bf37879 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:24.514174 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72751271-7e5b-4777-8499-246d890b0d15 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:24.525164 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:24.581918 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf163f02-579f-4980-8e36-2f48249eef67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:24.582548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf163f02-579f-4980-8e36-2f48249eef67 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:24.582824 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf163f02-579f-4980-8e36-2f48249eef67 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:24.583094 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf163f02-579f-4980-8e36-2f48249eef67 None admin] PATH_INFO: `/v3/services/network` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:24.583683 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf163f02-579f-4980-8e36-2f48249eef67 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:24.585154 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf163f02-579f-4980-8e36-2f48249eef67 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:24.600372 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-cf163f02-579f-4980-8e36-2f48249eef67 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 22:29:24.600372 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:24.600372 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:29:24.600372 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:29:24.600372 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:29:24.600372 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:29:24.600372 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:29:24.600372 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:29:24.600372 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 22:29:24.600372 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:24.600372 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:29:24.600372 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:24.600372 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:24.600372 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:24.600372 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:24.600372 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:24.600372 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:24.601466 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:24.601466 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:24.601466 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:24.601466 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:24.601466 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:24.601466 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:24.601466 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:29:24.601466 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:29:24.601466 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:29:24.601466 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:29:24.601466 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:24.601466 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:24.601466 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:24.601466 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:24.601466 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:24.601466 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:24.602763 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:24.602763 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:24.602763 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:24.602763 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:24.602763 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:24.602763 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:24.602763 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:24.602763 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:24.602763 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:24.602763 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:24.602763 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:29:24.602763 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:29:24.602763 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 22:29:24.602763 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:24.602763 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 347/710] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:29:24 2022] GET /identity/v3/services/network => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:24.603579 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f1d9e-7ca4-4e10-902a-297a91153abc None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:24.603579 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f1d9e-7ca4-4e10-902a-297a91153abc None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:24.603719 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f1d9e-7ca4-4e10-902a-297a91153abc None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:24.612804 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 348/711] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:24 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:29:24.613638 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f1d9e-7ca4-4e10-902a-297a91153abc None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:24.622234 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:24.638548 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44c0b95-e5c8-437a-b38d-f8d84bf37879 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:24.676381 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e35bb66-51ce-4d51-b9c2-799fbe9664ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:24.677019 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e35bb66-51ce-4d51-b9c2-799fbe9664ee None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:24.677301 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e35bb66-51ce-4d51-b9c2-799fbe9664ee None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:24.677576 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e35bb66-51ce-4d51-b9c2-799fbe9664ee None admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:24.678307 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e35bb66-51ce-4d51-b9c2-799fbe9664ee None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:24.678612 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e35bb66-51ce-4d51-b9c2-799fbe9664ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:24.680037 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e35bb66-51ce-4d51-b9c2-799fbe9664ee None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:24.699868 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 364/712] 10.222.0.52 () {56 vars in 1133 bytes} [Mon Aug 29 22:29:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:24.701170 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45ba9f-3e57-4512-bf77-15ed02c01df3 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:24.701452 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45ba9f-3e57-4512-bf77-15ed02c01df3 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:24.701724 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45ba9f-3e57-4512-bf77-15ed02c01df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:24.720371 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-af45ba9f-3e57-4512-bf77-15ed02c01df3 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:24.722221 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-af45ba9f-3e57-4512-bf77-15ed02c01df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:24.738369 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f1d9e-7ca4-4e10-902a-297a91153abc None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:24.780138 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68d0230-6a33-47d8-92b9-236507e080c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:24.780710 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d0230-6a33-47d8-92b9-236507e080c7 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:24.781015 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d0230-6a33-47d8-92b9-236507e080c7 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:24.781284 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d0230-6a33-47d8-92b9-236507e080c7 None admin] PATH_INFO: `/v3/services` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:24.782011 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68d0230-6a33-47d8-92b9-236507e080c7 None admin] RBAC: Adding query filter params (name=network) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:24.782314 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68d0230-6a33-47d8-92b9-236507e080c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:24.784542 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68d0230-6a33-47d8-92b9-236507e080c7 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:24.794987 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 349/713] 10.222.0.52 () {56 vars in 1112 bytes} [Mon Aug 29 22:29:24 2022] GET /identity/v3/services?name=network => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:24.795591 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 365/714] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:29:24.796149 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68d0230-6a33-47d8-92b9-236507e080c7 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:24.798671 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af45ba9f-3e57-4512-bf77-15ed02c01df3 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:24.849040 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:24.880509 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:24.906847 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af45ba9f-3e57-4512-bf77-15ed02c01df3 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:24.921834 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68d0230-6a33-47d8-92b9-236507e080c7 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:24.943766 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c73580-857c-44b8-8c1f-88a67bb7f036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:24.944601 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c73580-857c-44b8-8c1f-88a67bb7f036 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:24.946383 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c73580-857c-44b8-8c1f-88a67bb7f036 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:24.946383 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c73580-857c-44b8-8c1f-88a67bb7f036 None admin] PATH_INFO: `/v3/services` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:24.946383 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c73580-857c-44b8-8c1f-88a67bb7f036 None admin] RBAC: Adding query filter params (type=network) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:24.946383 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c73580-857c-44b8-8c1f-88a67bb7f036 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:24.947665 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c73580-857c-44b8-8c1f-88a67bb7f036 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:24.952683 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 366/715] 10.222.0.52 () {56 vars in 1112 bytes} [Mon Aug 29 22:29:24 2022] GET /identity/v3/services?type=network => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:24.973649 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3bab2-5f80-4cf4-bb85-b4815ec37fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:24.974284 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3bab2-5f80-4cf4-bb85-b4815ec37fcc None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:24.974562 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3bab2-5f80-4cf4-bb85-b4815ec37fcc None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:24.974835 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3bab2-5f80-4cf4-bb85-b4815ec37fcc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:24.985545 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dacbc41-5493-471f-9406-3f1ab0f498ac None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:24.985875 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dacbc41-5493-471f-9406-3f1ab0f498ac None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:24.986093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dacbc41-5493-471f-9406-3f1ab0f498ac None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:24.988006 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 367/716] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:29:24.993366 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd1ba8f-c7ac-46c7-bff8-c41ed3a9c44b None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:24.993703 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd1ba8f-c7ac-46c7-bff8-c41ed3a9c44b None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:24.994038 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd1ba8f-c7ac-46c7-bff8-c41ed3a9c44b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:25.027960 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-4bd1ba8f-c7ac-46c7-bff8-c41ed3a9c44b None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:25.037585 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-4bd1ba8f-c7ac-46c7-bff8-c41ed3a9c44b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:25.057964 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e3bab2-5f80-4cf4-bb85-b4815ec37fcc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:25.059476 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e3bab2-5f80-4cf4-bb85-b4815ec37fcc None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:25.102348 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:25.147651 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 368/717] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 22:29:25.191776 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 350/718] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:29:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:29:25.197319 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd1ba8f-c7ac-46c7-bff8-c41ed3a9c44b None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:25.198693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3bab2-5f80-4cf4-bb85-b4815ec37fcc None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:25.270367 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:25.307416 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd1ba8f-c7ac-46c7-bff8-c41ed3a9c44b None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:25.309065 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3bab2-5f80-4cf4-bb85-b4815ec37fcc None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:25.341449 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:25.343348 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a8b84-6197-4733-822c-9fa4761f41d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:25.344371 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a8b84-6197-4733-822c-9fa4761f41d2 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:25.344577 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a8b84-6197-4733-822c-9fa4761f41d2 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:25.344862 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a8b84-6197-4733-822c-9fa4761f41d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:25.347575 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeecd0ca-7742-43e2-9ccb-c9cd3ede9d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:25.348213 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeecd0ca-7742-43e2-9ccb-c9cd3ede9d60 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:25.348591 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeecd0ca-7742-43e2-9ccb-c9cd3ede9d60 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:25.348910 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeecd0ca-7742-43e2-9ccb-c9cd3ede9d60 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:25.357020 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6a8b84-6197-4733-822c-9fa4761f41d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:25.358491 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6a8b84-6197-4733-822c-9fa4761f41d2 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:25.360108 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeecd0ca-7742-43e2-9ccb-c9cd3ede9d60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:29:25.367648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-eeecd0ca-7742-43e2-9ccb-c9cd3ede9d60 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:29:25.367648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:25.367648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:25.367648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:25.367648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:25.367648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:25.367648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:25.367648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:25.367648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:25.367648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:25.367648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:25.367648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:25.367648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:29:25.367648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:29:25.367648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:29:25.367648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:29:25.369948 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:29:25.369948 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:29:25.369948 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:29:25.369948 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:29:25.369948 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:25.369948 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:25.369948 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:25.369948 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:25.369948 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:25.369948 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:25.369948 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:25.369948 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:25.369948 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:25.369948 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:25.369948 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:25.369948 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:25.372049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:25.372049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:25.372049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:25.372049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:25.372049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:29:25.372049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:29:25.372049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:29:25.372049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:29:25.372049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:29:25.372049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:29:25.372049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:29:25.372049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:25.372049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 351/719] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Aug 29 22:29:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:25.380525 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-5d6a8b84-6197-4733-822c-9fa4761f41d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:25.380525 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:25.380525 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:25.380525 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:25.380525 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:25.380525 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:25.380525 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:25.380525 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:25.380525 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:25.380525 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:25.380525 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:25.380525 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:25.380525 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:25.380525 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:25.380525 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:25.380525 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:25.382891 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:25.382891 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:25.382891 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:25.382891 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:25.382891 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:25.382891 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:25.382891 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:25.382891 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:25.382891 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:25.382891 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:25.382891 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:25.382891 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:25.382891 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:25.382891 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:25.382891 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:25.382891 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:25.382891 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:25.384899 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:25.384899 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:25.384899 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:25.385084 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeecd0ca-7742-43e2-9ccb-c9cd3ede9d60 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:25.385159 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 369/720] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:25.395945 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a8b84-6197-4733-822c-9fa4761f41d2 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:25.499331 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeecd0ca-7742-43e2-9ccb-c9cd3ede9d60 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:25.535540 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a8b84-6197-4733-822c-9fa4761f41d2 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:25.536083 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fa8119-6179-4353-b6cb-7eeaea6c5bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:25.536696 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa8119-6179-4353-b6cb-7eeaea6c5bb1 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:25.537001 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa8119-6179-4353-b6cb-7eeaea6c5bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:25.537270 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa8119-6179-4353-b6cb-7eeaea6c5bb1 None admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:25.537938 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fa8119-6179-4353-b6cb-7eeaea6c5bb1 None admin] RBAC: Adding query filter params (name=service) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:25.538234 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fa8119-6179-4353-b6cb-7eeaea6c5bb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:25.539866 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fa8119-6179-4353-b6cb-7eeaea6c5bb1 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:25.550872 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 352/721] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Aug 29 22:29:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:29:25.551775 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fa8119-6179-4353-b6cb-7eeaea6c5bb1 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:25.562337 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:25.576485 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191fc840-c25c-488b-a3cb-e59fb18d5ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:25.577333 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191fc840-c25c-488b-a3cb-e59fb18d5ea3 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:25.577762 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191fc840-c25c-488b-a3cb-e59fb18d5ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:25.578038 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191fc840-c25c-488b-a3cb-e59fb18d5ea3 None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:25.578678 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191fc840-c25c-488b-a3cb-e59fb18d5ea3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:25.578977 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191fc840-c25c-488b-a3cb-e59fb18d5ea3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:25.580928 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191fc840-c25c-488b-a3cb-e59fb18d5ea3 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:25.592886 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 370/722] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:29:25.594169 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805e2e4-d5ce-4d70-9f36-8000f8744e36 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:25.594454 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805e2e4-d5ce-4d70-9f36-8000f8744e36 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:25.594726 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805e2e4-d5ce-4d70-9f36-8000f8744e36 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:25.600656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 371/723] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:29:25.600656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2805e2e4-d5ce-4d70-9f36-8000f8744e36 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:25.621072 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:25.668468 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fa8119-6179-4353-b6cb-7eeaea6c5bb1 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:25.707027 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7fb059-dd21-4800-a9c8-436edf911139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:25.707663 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7fb059-dd21-4800-a9c8-436edf911139 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:25.707939 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7fb059-dd21-4800-a9c8-436edf911139 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:25.708207 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7fb059-dd21-4800-a9c8-436edf911139 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:25.713843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2805e2e4-d5ce-4d70-9f36-8000f8744e36 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:25.719521 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7fb059-dd21-4800-a9c8-436edf911139 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:29:25.724966 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-8e7fb059-dd21-4800-a9c8-436edf911139 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:29:25.724966 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:25.724966 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:25.724966 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:25.724966 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:25.724966 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:25.724966 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:25.724966 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:25.724966 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:25.724966 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:25.724966 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:25.724966 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:25.724966 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:29:25.724966 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:29:25.724966 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:29:25.724966 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:29:25.725886 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:29:25.725886 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:29:25.725886 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:29:25.725886 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:29:25.725886 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:25.725886 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:25.725886 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:25.725886 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:25.725886 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:25.725886 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:25.725886 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:25.725886 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:25.725886 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:25.725886 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:25.725886 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:25.725886 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:25.726737 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:25.726737 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:25.726737 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:25.726737 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:25.726737 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:29:25.726737 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:29:25.726737 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:29:25.726737 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:29:25.726737 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:29:25.726737 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:29:25.726737 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:29:25.726737 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:25.726737 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 353/724] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Aug 29 22:29:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:25.727435 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7fb059-dd21-4800-a9c8-436edf911139 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:25.758245 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65f819-5ccb-45d0-9c61-1f6b477d614d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:25.758797 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65f819-5ccb-45d0-9c61-1f6b477d614d None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:25.759070 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65f819-5ccb-45d0-9c61-1f6b477d614d None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:25.759398 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65f819-5ccb-45d0-9c61-1f6b477d614d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:25.773223 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce65f819-5ccb-45d0-9c61-1f6b477d614d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:25.774696 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce65f819-5ccb-45d0-9c61-1f6b477d614d None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:25.790945 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-ce65f819-5ccb-45d0-9c61-1f6b477d614d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:25.790945 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:25.790945 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:25.790945 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:25.790945 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:25.790945 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:25.790945 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:25.790945 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:25.790945 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:25.790945 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:25.790945 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:25.790945 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:25.790945 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:25.790945 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:25.790945 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:25.790945 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:25.791956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:25.791956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:25.791956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:25.791956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:25.791956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:25.791956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:25.791956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:25.791956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:25.791956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:25.791956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:25.791956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:25.791956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:25.791956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:25.791956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:25.791956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:25.791956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:25.791956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:25.793541 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:25.793541 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:25.793541 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:25.793541 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 372/725] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:25.793541 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65c6cf-d4f3-4c53-90af-61a907621483 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:25.793842 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65c6cf-d4f3-4c53-90af-61a907621483 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:25.793842 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65c6cf-d4f3-4c53-90af-61a907621483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:25.812436 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-4f65c6cf-d4f3-4c53-90af-61a907621483 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:25.813592 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-4f65c6cf-d4f3-4c53-90af-61a907621483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:25.842684 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:25.846988 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7fb059-dd21-4800-a9c8-436edf911139 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:25.861852 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:25.883831 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c776d6-eb6b-4199-bcd2-cf0ad97a3f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:25.884460 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c776d6-eb6b-4199-bcd2-cf0ad97a3f37 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:25.884787 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c776d6-eb6b-4199-bcd2-cf0ad97a3f37 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:25.885023 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c776d6-eb6b-4199-bcd2-cf0ad97a3f37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:25.886956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 373/726] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:29:25.894279 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f65c6cf-d4f3-4c53-90af-61a907621483 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:25.898554 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c776d6-eb6b-4199-bcd2-cf0ad97a3f37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:25.900023 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c776d6-eb6b-4199-bcd2-cf0ad97a3f37 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:25.912323 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-29c776d6-eb6b-4199-bcd2-cf0ad97a3f37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:25.912323 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:25.912323 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:25.912323 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:25.912323 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:25.912323 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:25.912323 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:25.912323 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:25.912323 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:25.912323 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:25.912323 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:25.912323 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:25.912323 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:29:25.912323 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:29:25.912323 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:29:25.913397 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:29:25.913397 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:25.913397 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:25.913397 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:25.913397 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:25.913397 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:25.913397 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:25.913397 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:25.913397 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:25.913397 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:25.913397 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:25.913397 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:25.913397 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:25.913397 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:25.913397 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:25.913397 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:25.913397 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:25.914288 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:29:25.914288 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:29:25.914288 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:29:25.914288 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:29:25.914288 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:29:25.914288 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:29:25.914288 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:25.914288 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:25.918234 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 354/727] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:29:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:25.919121 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c776d6-eb6b-4199-bcd2-cf0ad97a3f37 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:26.018764 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f65c6cf-d4f3-4c53-90af-61a907621483 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:26.032044 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c776d6-eb6b-4199-bcd2-cf0ad97a3f37 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:26.055961 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d1371-732e-4c2a-90ec-ef3fdea35ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:26.056442 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d1371-732e-4c2a-90ec-ef3fdea35ddc None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:26.056819 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d1371-732e-4c2a-90ec-ef3fdea35ddc None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:26.057169 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d1371-732e-4c2a-90ec-ef3fdea35ddc None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:26.057896 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642d1371-732e-4c2a-90ec-ef3fdea35ddc None admin] RBAC: Adding query filter params (name=admin) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:26.058273 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642d1371-732e-4c2a-90ec-ef3fdea35ddc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:26.059944 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642d1371-732e-4c2a-90ec-ef3fdea35ddc None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:26.072932 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 374/728] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:29:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:26.074034 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d1371-732e-4c2a-90ec-ef3fdea35ddc None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:26.081801 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546a498c-1a77-4e41-84a1-df30f36d986d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:26.082412 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a498c-1a77-4e41-84a1-df30f36d986d None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:26.082691 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a498c-1a77-4e41-84a1-df30f36d986d None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:26.082962 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a498c-1a77-4e41-84a1-df30f36d986d None admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:26.085539 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546a498c-1a77-4e41-84a1-df30f36d986d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:26.086060 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546a498c-1a77-4e41-84a1-df30f36d986d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:26.087663 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546a498c-1a77-4e41-84a1-df30f36d986d None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:26.101705 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 355/729] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:26.102966 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546a498c-1a77-4e41-84a1-df30f36d986d None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:26.117540 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:26.151386 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:26.215259 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d1371-732e-4c2a-90ec-ef3fdea35ddc None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:26.226124 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546a498c-1a77-4e41-84a1-df30f36d986d None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:26.254202 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a355a8a-e72a-46c7-ad5d-3308d8dbc7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:26.254808 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a355a8a-e72a-46c7-ad5d-3308d8dbc7c7 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:26.255089 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a355a8a-e72a-46c7-ad5d-3308d8dbc7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:26.255356 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a355a8a-e72a-46c7-ad5d-3308d8dbc7c7 None admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:26.256041 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a355a8a-e72a-46c7-ad5d-3308d8dbc7c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:26.256342 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a355a8a-e72a-46c7-ad5d-3308d8dbc7c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:26.257793 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a355a8a-e72a-46c7-ad5d-3308d8dbc7c7 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:26.264747 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556019bb-2959-4296-ab5a-ad4f5cb26843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:26.265709 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556019bb-2959-4296-ab5a-ad4f5cb26843 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:26.266189 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556019bb-2959-4296-ab5a-ad4f5cb26843 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:26.266633 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556019bb-2959-4296-ab5a-ad4f5cb26843 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:26.271858 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 375/730] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Aug 29 22:29:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:26.272799 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a355a8a-e72a-46c7-ad5d-3308d8dbc7c7 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:26.278632 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556019bb-2959-4296-ab5a-ad4f5cb26843 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:26.280339 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556019bb-2959-4296-ab5a-ad4f5cb26843 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:26.296999 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-556019bb-2959-4296-ab5a-ad4f5cb26843 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:26.296999 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:26.296999 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:26.296999 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:26.296999 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:26.296999 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:26.296999 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:26.296999 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:26.296999 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:26.296999 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:26.296999 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:26.296999 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:26.296999 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:26.296999 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:26.296999 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:26.296999 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:26.298023 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:26.298023 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:26.298023 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:26.298023 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:26.298023 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:26.298023 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:26.298023 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:26.298023 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:26.298023 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:26.298023 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:26.298023 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:26.298023 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:26.298023 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:26.298023 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:26.298023 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:26.298023 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:26.298023 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:26.304355 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:26.304355 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:26.304355 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:26.304355 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 356/731] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:26.304355 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556019bb-2959-4296-ab5a-ad4f5cb26843 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:26.375500 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:26.397088 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a355a8a-e72a-46c7-ad5d-3308d8dbc7c7 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:26.403397 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:26.419186 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556019bb-2959-4296-ab5a-ad4f5cb26843 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:26.432729 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed7899d-0929-4558-9b84-39fb723850d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:26.433382 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7899d-0929-4558-9b84-39fb723850d0 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:26.433664 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7899d-0929-4558-9b84-39fb723850d0 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:26.433939 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7899d-0929-4558-9b84-39fb723850d0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:26.441420 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed7899d-0929-4558-9b84-39fb723850d0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:26.442852 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed7899d-0929-4558-9b84-39fb723850d0 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:26.454621 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-aed7899d-0929-4558-9b84-39fb723850d0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 22:29:26.454621 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:26.454621 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:26.454621 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:26.454621 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:26.454621 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:26.454621 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:26.454621 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:26.454621 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:26.454621 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:26.454621 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:26.454621 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:26.454621 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:29:26.454621 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:29:26.454621 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:29:26.454621 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:29:26.456241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:26.456241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:26.456241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:29:26.456241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:26.456241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:29:26.456241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:26.456241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:26.456241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:26.456241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:26.456241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:26.456241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:26.456241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:26.456241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:26.456241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:26.456241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:26.456241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:26.456241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:26.456241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:26.457216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:26.457216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:26.457216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:29:26.457216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:29:26.457216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:29:26.457216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:29:26.457216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:29:26.457216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:29:26.457216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 22:29:26.457216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:26.457216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 376/732] 10.222.0.52 () {56 vars in 1117 bytes} [Mon Aug 29 22:29:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:26.457821 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed7899d-0929-4558-9b84-39fb723850d0 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:26.462983 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb815e8-5770-4b53-bf2b-69c88a3789bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:26.463596 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb815e8-5770-4b53-bf2b-69c88a3789bd None admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:26.463877 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb815e8-5770-4b53-bf2b-69c88a3789bd None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:26.464152 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb815e8-5770-4b53-bf2b-69c88a3789bd None admin] PATH_INFO: `/v3/services` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:26.464839 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb815e8-5770-4b53-bf2b-69c88a3789bd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:26.466205 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb815e8-5770-4b53-bf2b-69c88a3789bd None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:26.546683 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 357/733] 10.222.0.52 () {60 vars in 1141 bytes} [Mon Aug 29 22:29:26 2022] POST /identity/v3/services => generated 251 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:29:26.547514 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb815e8-5770-4b53-bf2b-69c88a3789bd None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:26.587508 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed7899d-0929-4558-9b84-39fb723850d0 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:26.655707 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4844e11-fac0-4d8c-8948-f69a13b2dd51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:26.656819 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4844e11-fac0-4d8c-8948-f69a13b2dd51 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:26.658960 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4844e11-fac0-4d8c-8948-f69a13b2dd51 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:26.659411 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4844e11-fac0-4d8c-8948-f69a13b2dd51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:26.660525 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4844e11-fac0-4d8c-8948-f69a13b2dd51 None admin] RBAC: Adding query filter params (role.id=882f35bd847a406eb2222aeafc517a46, scope.project.id=ee84c3340f2641ad963655a7a212ef96, user.id=bdbf961e02934e88af3a72d6218f32fa) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:26.661018 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4844e11-fac0-4d8c-8948-f69a13b2dd51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:26.662770 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4844e11-fac0-4d8c-8948-f69a13b2dd51 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:26.675886 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 377/734] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:29:26 2022] GET /identity/v3/role_assignments?user.id=bdbf961e02934e88af3a72d6218f32fa&scope.project.id=ee84c3340f2641ad963655a7a212ef96&role.id=882f35bd847a406eb2222aeafc517a46 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:26.675886 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4844e11-fac0-4d8c-8948-f69a13b2dd51 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:26.688636 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:26.725718 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb815e8-5770-4b53-bf2b-69c88a3789bd None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:26.768073 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e803d-df70-45a6-b02b-d8c1910f607a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:26.769520 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e803d-df70-45a6-b02b-d8c1910f607a None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:26.769985 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e803d-df70-45a6-b02b-d8c1910f607a None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:26.770361 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e803d-df70-45a6-b02b-d8c1910f607a None admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:26.772463 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47e803d-df70-45a6-b02b-d8c1910f607a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:26.772900 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47e803d-df70-45a6-b02b-d8c1910f607a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:26.774474 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47e803d-df70-45a6-b02b-d8c1910f607a None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:26.790986 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 358/735] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:26.796511 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e803d-df70-45a6-b02b-d8c1910f607a None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:26.812360 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:26.824228 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4844e11-fac0-4d8c-8948-f69a13b2dd51 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:26.875634 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57730ac-c4fc-4aa6-8220-eef8cb84bdab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:26.876238 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57730ac-c4fc-4aa6-8220-eef8cb84bdab None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:26.876672 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57730ac-c4fc-4aa6-8220-eef8cb84bdab None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:26.877244 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57730ac-c4fc-4aa6-8220-eef8cb84bdab None admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:26.877741 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57730ac-c4fc-4aa6-8220-eef8cb84bdab None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:26.878046 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57730ac-c4fc-4aa6-8220-eef8cb84bdab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:26.880130 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57730ac-c4fc-4aa6-8220-eef8cb84bdab None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:26.904881 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 378/736] 10.222.0.52 () {56 vars in 1149 bytes} [Mon Aug 29 22:29:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:26.909186 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57730ac-c4fc-4aa6-8220-eef8cb84bdab None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:26.934873 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e803d-df70-45a6-b02b-d8c1910f607a None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:26.977687 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db4e81-fd37-4d60-aebd-9ed6de3eab33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:26.978385 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db4e81-fd37-4d60-aebd-9ed6de3eab33 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:26.978714 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db4e81-fd37-4d60-aebd-9ed6de3eab33 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:26.978991 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db4e81-fd37-4d60-aebd-9ed6de3eab33 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:26.987733 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db4e81-fd37-4d60-aebd-9ed6de3eab33 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:26.989394 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db4e81-fd37-4d60-aebd-9ed6de3eab33 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:26.991896 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:27.002357 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-d7db4e81-fd37-4d60-aebd-9ed6de3eab33 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 22:29:27.002357 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:27.002357 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:27.002357 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:27.002357 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:27.002357 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:27.002357 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:27.002357 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:27.002357 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:27.002357 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:27.002357 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:27.002357 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:27.002357 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:29:27.002357 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:29:27.002357 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:29:27.002357 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:29:27.003455 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:27.003455 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:27.003455 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:29:27.003455 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:27.003455 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:29:27.003455 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:27.003455 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:27.003455 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:27.003455 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:27.003455 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:27.003455 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:27.003455 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:27.003455 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:27.003455 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:27.003455 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:27.003455 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:27.003455 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:27.003455 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:27.005069 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:27.005069 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:27.005069 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:29:27.005069 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:29:27.005069 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:29:27.005069 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:29:27.005069 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:29:27.005069 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:29:27.005069 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 22:29:27.005069 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:27.005069 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 359/737] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Aug 29 22:29:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:29:27.009360 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db4e81-fd37-4d60-aebd-9ed6de3eab33 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:27.055029 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57730ac-c4fc-4aa6-8220-eef8cb84bdab None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:27.103865 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9185f154-4582-4cce-ae83-b39cf789f011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:27.105968 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9185f154-4582-4cce-ae83-b39cf789f011 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:27.106467 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9185f154-4582-4cce-ae83-b39cf789f011 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:27.110168 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9185f154-4582-4cce-ae83-b39cf789f011 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:27.148440 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:27.225315 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9185f154-4582-4cce-ae83-b39cf789f011 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:27.227230 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9185f154-4582-4cce-ae83-b39cf789f011 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:27.248465 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db4e81-fd37-4d60-aebd-9ed6de3eab33 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:27.293192 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add3d99a-3224-4c00-8425-2ba7c772359a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:27.293796 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add3d99a-3224-4c00-8425-2ba7c772359a None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:27.294072 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add3d99a-3224-4c00-8425-2ba7c772359a None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:27.294344 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add3d99a-3224-4c00-8425-2ba7c772359a None admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:27.295287 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add3d99a-3224-4c00-8425-2ba7c772359a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:27.295414 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add3d99a-3224-4c00-8425-2ba7c772359a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:27.296788 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add3d99a-3224-4c00-8425-2ba7c772359a None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:27.304669 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:27.321072 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 360/738] 10.222.0.52 () {56 vars in 1135 bytes} [Mon Aug 29 22:29:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:29:27.322576 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf43e89-689c-4f92-b759-8c6f4923fc4d None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:27.322967 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf43e89-689c-4f92-b759-8c6f4923fc4d None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:27.323407 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf43e89-689c-4f92-b759-8c6f4923fc4d None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:27.325028 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 361/739] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:29:27.327615 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf43e89-689c-4f92-b759-8c6f4923fc4d None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:27.354516 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 379/740] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:29:26 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:29:27.356079 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ef2c8-99be-444d-8b92-96a43d7815d6 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:27.356376 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ef2c8-99be-444d-8b92-96a43d7815d6 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:27.356833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ef2c8-99be-444d-8b92-96a43d7815d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:27.379791 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-f72ef2c8-99be-444d-8b92-96a43d7815d6 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:27.381348 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-f72ef2c8-99be-444d-8b92-96a43d7815d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:27.451068 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf43e89-689c-4f92-b759-8c6f4923fc4d None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:27.473218 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 380/741] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:27 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:29:27.474156 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72ef2c8-99be-444d-8b92-96a43d7815d6 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:27.485998 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:27.500530 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeebf37-9010-4509-a084-e6d51757ab22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:27.501484 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeebf37-9010-4509-a084-e6d51757ab22 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:27.502313 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeebf37-9010-4509-a084-e6d51757ab22 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:27.502612 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeebf37-9010-4509-a084-e6d51757ab22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:27.590133 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeebf37-9010-4509-a084-e6d51757ab22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:27.592429 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeebf37-9010-4509-a084-e6d51757ab22 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:27.784682 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 362/742] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:29:27 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 22:29:27.789160 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:27.797660 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72ef2c8-99be-444d-8b92-96a43d7815d6 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:27.797790 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeebf37-9010-4509-a084-e6d51757ab22 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:27.844751 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3609ddc7-73bb-4a63-bfd5-9375839c0fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:27.844751 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609ddc7-73bb-4a63-bfd5-9375839c0fc0 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:27.844751 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609ddc7-73bb-4a63-bfd5-9375839c0fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:27.844751 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609ddc7-73bb-4a63-bfd5-9375839c0fc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:27.854365 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:27.866103 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3609ddc7-73bb-4a63-bfd5-9375839c0fc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:27.867878 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3609ddc7-73bb-4a63-bfd5-9375839c0fc0 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:27.886133 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-3609ddc7-73bb-4a63-bfd5-9375839c0fc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:27.886133 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:27.886133 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:27.886133 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:27.886133 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:27.886133 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:27.886133 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:27.886133 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:27.886133 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:27.886133 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:27.886133 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:27.886133 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:27.886133 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:27.886133 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:27.886133 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:27.886133 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:27.887157 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:27.887157 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:27.887157 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:27.887157 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:27.887157 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:27.887157 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:27.887157 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:27.887157 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:27.887157 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:27.887157 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:27.887157 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:27.887157 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:27.887157 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:27.887157 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:27.887157 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:27.887157 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:27.887157 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:27.888065 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:27.888065 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:27.888065 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:27.890264 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 381/743] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 416 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:27.891383 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3609ddc7-73bb-4a63-bfd5-9375839c0fc0 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:27.943691 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeebf37-9010-4509-a084-e6d51757ab22 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:27.988754 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f18e55-ba81-48c6-9d00-c165c0c11583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:27.989419 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f18e55-ba81-48c6-9d00-c165c0c11583 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:27.989702 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f18e55-ba81-48c6-9d00-c165c0c11583 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:27.990011 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f18e55-ba81-48c6-9d00-c165c0c11583 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:28.002118 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f18e55-ba81-48c6-9d00-c165c0c11583 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:28.003695 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f18e55-ba81-48c6-9d00-c165c0c11583 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:28.014255 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3609ddc7-73bb-4a63-bfd5-9375839c0fc0 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:28.021771 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-95f18e55-ba81-48c6-9d00-c165c0c11583 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:28.021771 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:28.021771 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:28.021771 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:28.021771 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:28.021771 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:28.021771 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:28.021771 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:28.021771 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:28.021771 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:28.021771 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:28.021771 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:28.021771 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:28.021771 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:28.021771 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:28.021771 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:28.023605 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:28.023605 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:28.023605 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:28.023605 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:28.023605 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:28.023605 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:28.023605 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:28.023605 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:28.023605 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:28.023605 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:28.023605 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:28.023605 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:28.023605 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:28.023605 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:28.023605 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:28.023605 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:28.023605 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:28.024742 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:28.024742 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:28.024742 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:28.030696 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 363/744] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:28.031906 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f18e55-ba81-48c6-9d00-c165c0c11583 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:28.071028 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d3c2d-8a7c-47c2-90be-43fce4c6927d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:28.071565 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d3c2d-8a7c-47c2-90be-43fce4c6927d None admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:28.071899 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d3c2d-8a7c-47c2-90be-43fce4c6927d None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:28.072448 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d3c2d-8a7c-47c2-90be-43fce4c6927d None admin] PATH_INFO: `/v3/services` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:28.073393 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0d3c2d-8a7c-47c2-90be-43fce4c6927d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:28.075658 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0d3c2d-8a7c-47c2-90be-43fce4c6927d None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:28.119569 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:28.142528 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f18e55-ba81-48c6-9d00-c165c0c11583 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:28.148185 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 382/745] 10.222.0.52 () {60 vars in 1141 bytes} [Mon Aug 29 22:29:27 2022] POST /identity/v3/services => generated 240 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:29:28.149463 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d3c2d-8a7c-47c2-90be-43fce4c6927d None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:28.154376 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:28.215326 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f7c2a-47f3-4642-a139-218500d29917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:28.215956 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f7c2a-47f3-4642-a139-218500d29917 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:28.216219 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f7c2a-47f3-4642-a139-218500d29917 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:28.216498 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f7c2a-47f3-4642-a139-218500d29917 None admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:28.217194 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669f7c2a-47f3-4642-a139-218500d29917 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:28.217597 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669f7c2a-47f3-4642-a139-218500d29917 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:28.218900 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669f7c2a-47f3-4642-a139-218500d29917 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:28.247058 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 364/746] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:28.253280 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f7c2a-47f3-4642-a139-218500d29917 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:28.295769 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d3c2d-8a7c-47c2-90be-43fce4c6927d None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:28.342087 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3f916b-d11d-45a8-974c-0a80fdf2305f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:28.343552 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f916b-d11d-45a8-974c-0a80fdf2305f None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:28.344148 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f916b-d11d-45a8-974c-0a80fdf2305f None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:28.344607 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f916b-d11d-45a8-974c-0a80fdf2305f None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:28.346297 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3f916b-d11d-45a8-974c-0a80fdf2305f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:28.346788 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3f916b-d11d-45a8-974c-0a80fdf2305f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:28.349297 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3f916b-d11d-45a8-974c-0a80fdf2305f None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:28.372373 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 383/747] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:28.373979 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3f916b-d11d-45a8-974c-0a80fdf2305f None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:28.438447 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f7c2a-47f3-4642-a139-218500d29917 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:28.466619 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:28.492986 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:28.501191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a1169-bce6-49db-b63d-ca9759589a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:28.502851 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a1169-bce6-49db-b63d-ca9759589a0e None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:28.503990 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a1169-bce6-49db-b63d-ca9759589a0e None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:28.504452 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a1169-bce6-49db-b63d-ca9759589a0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:28.514713 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675a1169-bce6-49db-b63d-ca9759589a0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:28.517099 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675a1169-bce6-49db-b63d-ca9759589a0e None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:28.518192 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3f916b-d11d-45a8-974c-0a80fdf2305f None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:28.533417 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-675a1169-bce6-49db-b63d-ca9759589a0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:28.533417 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:28.533417 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:28.533417 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:28.533417 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:28.533417 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:28.533417 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:28.533417 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:28.533417 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:28.533417 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:28.533417 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:28.533417 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:28.533417 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:29:28.533417 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:29:28.533417 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:29:28.534654 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:29:28.534654 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:28.534654 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:28.534654 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:28.534654 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:28.534654 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:28.534654 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:28.534654 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:28.534654 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:28.534654 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:28.534654 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:28.534654 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:28.534654 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:28.534654 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:28.534654 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:28.534654 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:28.534654 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:28.535550 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:29:28.535550 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:29:28.535550 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:29:28.535550 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:29:28.535550 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:29:28.535550 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:29:28.535550 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:28.535550 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:28.544102 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 365/748] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:29:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:28.546205 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a1169-bce6-49db-b63d-ca9759589a0e None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:28.609544 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a28d9b9-04d6-4c31-a35b-1d23b468da42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:28.610113 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a28d9b9-04d6-4c31-a35b-1d23b468da42 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:28.610396 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a28d9b9-04d6-4c31-a35b-1d23b468da42 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:28.610679 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a28d9b9-04d6-4c31-a35b-1d23b468da42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:28.629898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a28d9b9-04d6-4c31-a35b-1d23b468da42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:28.631377 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a28d9b9-04d6-4c31-a35b-1d23b468da42 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:28.654667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-7a28d9b9-04d6-4c31-a35b-1d23b468da42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:28.654667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:28.654667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:28.654667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:28.654667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:28.654667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:28.654667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:28.654667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:28.654667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:28.654667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:28.654667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:28.654667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:28.654667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:29:28.654667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:29:28.654667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:29:28.655629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:29:28.655629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:28.655629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:28.655629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:28.655629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:28.655629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:28.655629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:28.655629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:28.655629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:28.655629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:28.655629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:28.655629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:28.655629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:28.655629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:28.655629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:28.655629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:28.655629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:28.656524 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:29:28.656524 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:29:28.656524 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:29:28.656524 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:29:28.656524 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:29:28.656524 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:29:28.656524 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:28.656524 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:28.656524 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 384/749] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:29:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:29:28.671953 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a1169-bce6-49db-b63d-ca9759589a0e None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:28.675828 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a28d9b9-04d6-4c31-a35b-1d23b468da42 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:28.718085 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c308560-85ae-4abb-89b3-6279b6642232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:28.720816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c308560-85ae-4abb-89b3-6279b6642232 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:28.721659 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c308560-85ae-4abb-89b3-6279b6642232 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:28.721958 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c308560-85ae-4abb-89b3-6279b6642232 None admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:28.728395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c308560-85ae-4abb-89b3-6279b6642232 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:28.728738 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c308560-85ae-4abb-89b3-6279b6642232 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:28.730237 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c308560-85ae-4abb-89b3-6279b6642232 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:28.764573 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 366/750] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Aug 29 22:29:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:28.771374 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c308560-85ae-4abb-89b3-6279b6642232 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:28.847683 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:28.849702 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a28d9b9-04d6-4c31-a35b-1d23b468da42 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:28.905580 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c308560-85ae-4abb-89b3-6279b6642232 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:28.907242 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:28.921111 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d131ab-29f7-44d0-879c-201cb61c6211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:28.921778 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d131ab-29f7-44d0-879c-201cb61c6211 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:28.921983 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d131ab-29f7-44d0-879c-201cb61c6211 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:28.922250 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d131ab-29f7-44d0-879c-201cb61c6211 None admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:28.922977 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d131ab-29f7-44d0-879c-201cb61c6211 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:28.923229 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d131ab-29f7-44d0-879c-201cb61c6211 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:28.925125 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d131ab-29f7-44d0-879c-201cb61c6211 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:28.943192 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 385/751] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Aug 29 22:29:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:29:28.943192 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85588a01-d594-488e-aaa4-61604135da4d None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:28.943192 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85588a01-d594-488e-aaa4-61604135da4d None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:28.943192 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85588a01-d594-488e-aaa4-61604135da4d None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:28.946241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 386/752] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:29:28.947529 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e52b1ec-8955-4adf-ae08-573c0a405385 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:28.947727 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e52b1ec-8955-4adf-ae08-573c0a405385 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:28.948000 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e52b1ec-8955-4adf-ae08-573c0a405385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:28.958991 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e6e5c-6953-450a-88f4-7de0f0c0b138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:28.959855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e6e5c-6953-450a-88f4-7de0f0c0b138 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:28.960282 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e6e5c-6953-450a-88f4-7de0f0c0b138 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:28.960786 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e6e5c-6953-450a-88f4-7de0f0c0b138 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:28.961802 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92e6e5c-6953-450a-88f4-7de0f0c0b138 None admin] RBAC: Adding query filter params (role.id=db83c2f8e21945c3ab9642c620d29a23, scope.project.id=ee84c3340f2641ad963655a7a212ef96, user.id=f77dfe78b1a8405084c06d9acee96511) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:28.962270 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92e6e5c-6953-450a-88f4-7de0f0c0b138 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:28.963989 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92e6e5c-6953-450a-88f4-7de0f0c0b138 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:28.967355 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-1e52b1ec-8955-4adf-ae08-573c0a405385 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:28.968547 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-1e52b1ec-8955-4adf-ae08-573c0a405385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:28.971396 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 367/753] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:29:28 2022] GET /identity/v3/role_assignments?user.id=f77dfe78b1a8405084c06d9acee96511&scope.project.id=ee84c3340f2641ad963655a7a212ef96&role.id=db83c2f8e21945c3ab9642c620d29a23 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:29:28.972581 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e6e5c-6953-450a-88f4-7de0f0c0b138 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:29.074117 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 387/754] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:29:29.074117 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e9549-34e5-490d-b8a9-14b87e6c97a1 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:29.074117 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e9549-34e5-490d-b8a9-14b87e6c97a1 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:29.074117 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e9549-34e5-490d-b8a9-14b87e6c97a1 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:29.077555 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 388/755] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:29:29.079300 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c53212-9ebb-4f0f-b7b9-a7a95e9c945a None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:29.079588 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c53212-9ebb-4f0f-b7b9-a7a95e9c945a None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:29.079865 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c53212-9ebb-4f0f-b7b9-a7a95e9c945a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:29.102940 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-68c53212-9ebb-4f0f-b7b9-a7a95e9c945a None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:29.104643 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-68c53212-9ebb-4f0f-b7b9-a7a95e9c945a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:29.135492 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e6e5c-6953-450a-88f4-7de0f0c0b138 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:29.162271 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:29.191098 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ce661-6681-4955-b5d7-f5d28f5eb876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:29.192108 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ce661-6681-4955-b5d7-f5d28f5eb876 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:29.192643 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ce661-6681-4955-b5d7-f5d28f5eb876 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:29.194055 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ce661-6681-4955-b5d7-f5d28f5eb876 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:29.198082 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626ce661-6681-4955-b5d7-f5d28f5eb876 None admin] RBAC: Adding query filter params (role.id=882f35bd847a406eb2222aeafc517a46, scope.project.id=ee84c3340f2641ad963655a7a212ef96, user.id=57c175ac90e545358746176972ff39a3) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:29.199645 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626ce661-6681-4955-b5d7-f5d28f5eb876 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:29.202807 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626ce661-6681-4955-b5d7-f5d28f5eb876 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:29.219292 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 368/756] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:29:28 2022] GET /identity/v3/role_assignments?user.id=57c175ac90e545358746176972ff39a3&scope.project.id=ee84c3340f2641ad963655a7a212ef96&role.id=882f35bd847a406eb2222aeafc517a46 => generated 259 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:29.226992 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 389/757] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:29:29.384141 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ce661-6681-4955-b5d7-f5d28f5eb876 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:29.515292 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ce661-6681-4955-b5d7-f5d28f5eb876 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:29.573996 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c817536b-ae80-4388-baef-22ce5c8e6fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:29.574632 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817536b-ae80-4388-baef-22ce5c8e6fdf None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:29.574963 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817536b-ae80-4388-baef-22ce5c8e6fdf None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:29.575275 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817536b-ae80-4388-baef-22ce5c8e6fdf None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:29.575820 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c817536b-ae80-4388-baef-22ce5c8e6fdf None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:29.577234 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c817536b-ae80-4388-baef-22ce5c8e6fdf None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:29.589855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-c817536b-ae80-4388-baef-22ce5c8e6fdf None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 22:29:29.589855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:29.589855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:29:29.589855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:29:29.589855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:29:29.589855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:29:29.589855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:29:29.589855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:29:29.589855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 22:29:29.589855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:29.589855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:29:29.589855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:29.589855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:29.589855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:29.589855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:29.589855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:29.589855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:29.590927 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:29.590927 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:29.590927 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:29.590927 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:29.590927 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:29.590927 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:29.590927 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:29:29.590927 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:29:29.590927 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:29:29.590927 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:29:29.590927 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:29.590927 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:29.590927 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:29.590927 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:29.590927 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:29.590927 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:29.591778 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:29.591778 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:29.591778 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:29.591778 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:29.591778 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:29.591778 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:29.591778 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:29.591778 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:29.591778 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:29.591778 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:29.591778 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:29:29.591778 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:29:29.591778 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 22:29:29.591778 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:29.591778 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 369/758] 10.222.0.52 () {56 vars in 1129 bytes} [Mon Aug 29 22:29:29 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:29.597012 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c53212-9ebb-4f0f-b7b9-a7a95e9c945a None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:29.604746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c817536b-ae80-4388-baef-22ce5c8e6fdf None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:29.686352 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:29.693129 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:29.796693 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c53212-9ebb-4f0f-b7b9-a7a95e9c945a None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:29.803393 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c817536b-ae80-4388-baef-22ce5c8e6fdf None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:29.858585 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3363964-2406-40e3-a9df-ffafdabca5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:29.859270 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3363964-2406-40e3-a9df-ffafdabca5f9 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:29.859497 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3363964-2406-40e3-a9df-ffafdabca5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:29.859776 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3363964-2406-40e3-a9df-ffafdabca5f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:29.872766 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3363964-2406-40e3-a9df-ffafdabca5f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:29:29.877547 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d4748b-dac9-49db-8dd2-d6305b87db7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:29.877916 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-a3363964-2406-40e3-a9df-ffafdabca5f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:29:29.877916 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:29.877916 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:29.877916 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:29.877916 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:29.877916 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:29.877916 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:29.877916 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:29.877916 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:29.877916 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:29.877916 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:29.877916 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:29.877916 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:29:29.877916 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:29:29.877916 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:29:29.877916 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:29:29.878822 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:29:29.878822 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:29:29.878822 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:29:29.878822 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:29:29.878822 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:29.878822 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:29.878822 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:29.878822 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:29.878822 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:29.878822 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:29.878822 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:29.878822 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:29.878822 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:29.878822 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:29.878822 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:29.878822 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:29.879835 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:29.879835 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:29.879835 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:29.879835 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:29.879835 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:29:29.879835 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:29:29.879835 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:29:29.879835 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:29:29.879835 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:29:29.879835 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:29:29.879835 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:29:29.879835 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:29.879835 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 390/759] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Aug 29 22:29:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:29.881335 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d4748b-dac9-49db-8dd2-d6305b87db7a None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:29.881804 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d4748b-dac9-49db-8dd2-d6305b87db7a None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:29.882234 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d4748b-dac9-49db-8dd2-d6305b87db7a None admin] PATH_INFO: `/v3/services` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:29.883037 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d4748b-dac9-49db-8dd2-d6305b87db7a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:29.883502 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d4748b-dac9-49db-8dd2-d6305b87db7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:29.889201 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d4748b-dac9-49db-8dd2-d6305b87db7a None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:29.893602 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3363964-2406-40e3-a9df-ffafdabca5f9 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:29.901548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 370/760] 10.222.0.52 () {56 vars in 1124 bytes} [Mon Aug 29 22:29:29 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:29:29.907021 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d4748b-dac9-49db-8dd2-d6305b87db7a None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:30.012903 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3363964-2406-40e3-a9df-ffafdabca5f9 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:30.015869 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:30.058958 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd47d20-2258-46e7-8513-6126ea1e0efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:30.059476 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd47d20-2258-46e7-8513-6126ea1e0efa None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:30.059751 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd47d20-2258-46e7-8513-6126ea1e0efa None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:30.060017 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd47d20-2258-46e7-8513-6126ea1e0efa None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:30.060678 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd47d20-2258-46e7-8513-6126ea1e0efa None admin] RBAC: Adding query filter params (name=admin) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:30.061770 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd47d20-2258-46e7-8513-6126ea1e0efa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:30.063867 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd47d20-2258-46e7-8513-6126ea1e0efa None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:30.090997 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 391/761] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:29:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:29:30.101826 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd47d20-2258-46e7-8513-6126ea1e0efa None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:30.124680 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d4748b-dac9-49db-8dd2-d6305b87db7a None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:30.141117 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:30.166398 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd30f0d2-32d8-41f2-bde6-3560cc1f9cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:30.167019 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30f0d2-32d8-41f2-bde6-3560cc1f9cce None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:30.167308 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30f0d2-32d8-41f2-bde6-3560cc1f9cce None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:30.167590 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30f0d2-32d8-41f2-bde6-3560cc1f9cce None admin] PATH_INFO: `/v3/services` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:30.168235 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd30f0d2-32d8-41f2-bde6-3560cc1f9cce None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:30.168586 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd30f0d2-32d8-41f2-bde6-3560cc1f9cce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:30.170383 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd30f0d2-32d8-41f2-bde6-3560cc1f9cce None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:30.177245 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 371/762] 10.222.0.52 () {56 vars in 1124 bytes} [Mon Aug 29 22:29:29 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:30.183874 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd30f0d2-32d8-41f2-bde6-3560cc1f9cce None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:30.331558 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd47d20-2258-46e7-8513-6126ea1e0efa None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:30.351850 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd30f0d2-32d8-41f2-bde6-3560cc1f9cce None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:30.394868 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1151f33-25cd-4d31-adab-eb8378a4d929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:30.395502 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1151f33-25cd-4d31-adab-eb8378a4d929 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:30.395785 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1151f33-25cd-4d31-adab-eb8378a4d929 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:30.396057 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1151f33-25cd-4d31-adab-eb8378a4d929 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:30.409913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1151f33-25cd-4d31-adab-eb8378a4d929 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:30.411695 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1151f33-25cd-4d31-adab-eb8378a4d929 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:30.428499 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-c1151f33-25cd-4d31-adab-eb8378a4d929 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:30.428499 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:30.428499 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:30.428499 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:30.428499 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:30.428499 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:30.428499 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:30.428499 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:30.428499 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:30.428499 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:30.428499 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:30.428499 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:30.428499 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:30.428499 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:30.428499 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:30.428499 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:30.429551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:30.429551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:30.429551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:30.429551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:30.429551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:30.429551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:30.429551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:30.429551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:30.429551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:30.429551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:30.429551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:30.429551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:30.429551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:30.429551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:30.429551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:30.429551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:30.429551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:30.430877 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:30.430877 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:30.430877 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:30.430877 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 392/763] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:30.437151 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1151f33-25cd-4d31-adab-eb8378a4d929 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:30.449587 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47b48c4-d04d-4eb3-95cc-f4d68578568d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:30.451467 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b48c4-d04d-4eb3-95cc-f4d68578568d None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:30.451751 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b48c4-d04d-4eb3-95cc-f4d68578568d None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:30.452026 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b48c4-d04d-4eb3-95cc-f4d68578568d None admin] PATH_INFO: `/v3/endpoints` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:30.452765 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47b48c4-d04d-4eb3-95cc-f4d68578568d None admin] RBAC: Adding query filter params (interface=public, service_id=b489030ca7b04594ace8e51ad9c63d44, region_id=RegionOne) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:30.453079 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47b48c4-d04d-4eb3-95cc-f4d68578568d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:30.455775 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47b48c4-d04d-4eb3-95cc-f4d68578568d None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:30.473119 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:30.473964 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 372/764] 10.222.0.52 () {56 vars in 1251 bytes} [Mon Aug 29 22:29:30 2022] GET /identity/v3/endpoints?service_id=b489030ca7b04594ace8e51ad9c63d44&interface=public®ion_id=RegionOne => generated 194 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:30.480699 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47b48c4-d04d-4eb3-95cc-f4d68578568d None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:30.562198 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:30.614554 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1151f33-25cd-4d31-adab-eb8378a4d929 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:30.640927 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47b48c4-d04d-4eb3-95cc-f4d68578568d None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:30.657802 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b361187-89f3-41f9-a97e-b52ec70b80b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:30.658470 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b361187-89f3-41f9-a97e-b52ec70b80b5 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:30.658840 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b361187-89f3-41f9-a97e-b52ec70b80b5 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:30.659118 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b361187-89f3-41f9-a97e-b52ec70b80b5 None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:30.659873 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b361187-89f3-41f9-a97e-b52ec70b80b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:30.660183 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b361187-89f3-41f9-a97e-b52ec70b80b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:30.661704 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b361187-89f3-41f9-a97e-b52ec70b80b5 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:30.676633 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 393/765] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:30.677954 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87a8f3-5af2-4736-b9d9-b5a576014da4 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:30.678243 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87a8f3-5af2-4736-b9d9-b5a576014da4 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:30.678517 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87a8f3-5af2-4736-b9d9-b5a576014da4 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:30.679985 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 394/766] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:29:30.681572 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87a8f3-5af2-4736-b9d9-b5a576014da4 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:30.697027 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e56604-0828-4d31-a816-1f50c95ddc4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:30.697634 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e56604-0828-4d31-a816-1f50c95ddc4b None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:30.698212 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e56604-0828-4d31-a816-1f50c95ddc4b None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:30.698560 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e56604-0828-4d31-a816-1f50c95ddc4b None admin] PATH_INFO: `/v3/services` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:30.699647 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e56604-0828-4d31-a816-1f50c95ddc4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:30.701277 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e56604-0828-4d31-a816-1f50c95ddc4b None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:30.709630 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 373/767] 10.222.0.52 () {56 vars in 1087 bytes} [Mon Aug 29 22:29:30 2022] GET /identity/v3/services => generated 772 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:29:30.711653 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42111a44-3694-4ec8-8da8-e0828315c388 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:30.711959 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42111a44-3694-4ec8-8da8-e0828315c388 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:30.712239 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42111a44-3694-4ec8-8da8-e0828315c388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:30.735053 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-42111a44-3694-4ec8-8da8-e0828315c388 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:30.736931 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-42111a44-3694-4ec8-8da8-e0828315c388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:30.780595 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:30.834443 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87a8f3-5af2-4736-b9d9-b5a576014da4 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:30.874309 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 374/768] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:29:30.893589 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a3f5e8-0ab7-4c1e-ba45-73a434abcbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:30.894270 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a3f5e8-0ab7-4c1e-ba45-73a434abcbb4 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:30.894571 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a3f5e8-0ab7-4c1e-ba45-73a434abcbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:30.894848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a3f5e8-0ab7-4c1e-ba45-73a434abcbb4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:30.903230 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a3f5e8-0ab7-4c1e-ba45-73a434abcbb4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:30.904782 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a3f5e8-0ab7-4c1e-ba45-73a434abcbb4 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:30.917508 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-b8a3f5e8-0ab7-4c1e-ba45-73a434abcbb4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 22:29:30.917508 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:30.917508 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:30.917508 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:30.917508 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:30.917508 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:30.917508 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:30.917508 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:30.917508 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:30.917508 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:30.917508 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:30.917508 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:30.917508 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:29:30.917508 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:29:30.917508 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:29:30.917508 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:29:30.919805 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:30.919805 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:30.919805 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:29:30.919805 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:30.919805 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:29:30.919805 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:30.919805 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:30.919805 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:30.919805 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:30.919805 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:30.919805 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:30.919805 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:30.919805 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:30.919805 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:30.919805 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:30.919805 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:30.919805 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:30.919805 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:30.921621 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:30.921621 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:30.921621 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:29:30.921621 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:29:30.921621 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:29:30.921621 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:29:30.921621 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:29:30.921621 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:29:30.921621 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 22:29:30.921621 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:30.921621 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 395/769] 10.222.0.52 () {56 vars in 1093 bytes} [Mon Aug 29 22:29:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:30.930067 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42111a44-3694-4ec8-8da8-e0828315c388 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:30.972863 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:31.051109 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42111a44-3694-4ec8-8da8-e0828315c388 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:31.095891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db4b42b-de95-4d17-b39e-aaa83a350f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:31.096506 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4b42b-de95-4d17-b39e-aaa83a350f87 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:31.096818 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4b42b-de95-4d17-b39e-aaa83a350f87 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:31.097101 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4b42b-de95-4d17-b39e-aaa83a350f87 None admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:31.097849 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db4b42b-de95-4d17-b39e-aaa83a350f87 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:31.098160 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db4b42b-de95-4d17-b39e-aaa83a350f87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:31.099603 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db4b42b-de95-4d17-b39e-aaa83a350f87 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:31.120351 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 375/770] 10.222.0.52 () {56 vars in 1133 bytes} [Mon Aug 29 22:29:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:31.125914 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a3f5e8-0ab7-4c1e-ba45-73a434abcbb4 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:31.227777 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db4b42b-de95-4d17-b39e-aaa83a350f87 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:31.249809 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a3f5e8-0ab7-4c1e-ba45-73a434abcbb4 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:31.287319 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9cff0f-53dc-4cc3-b5c5-f37f938ceb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:31.287963 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cff0f-53dc-4cc3-b5c5-f37f938ceb39 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:31.288248 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cff0f-53dc-4cc3-b5c5-f37f938ceb39 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:31.288528 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cff0f-53dc-4cc3-b5c5-f37f938ceb39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:31.296791 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:31.302344 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9cff0f-53dc-4cc3-b5c5-f37f938ceb39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:31.303805 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9cff0f-53dc-4cc3-b5c5-f37f938ceb39 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:31.319874 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-5a9cff0f-53dc-4cc3-b5c5-f37f938ceb39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:31.319874 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:31.319874 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:31.319874 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:31.319874 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:31.319874 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:31.319874 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:31.319874 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:31.319874 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:31.319874 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:31.319874 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:31.319874 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:31.319874 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:31.319874 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:31.319874 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:31.319874 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:31.320920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:31.320920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:31.320920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:31.320920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:31.320920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:31.320920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:31.320920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:31.320920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:31.320920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:31.320920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:31.320920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:31.320920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:31.320920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:31.320920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:31.320920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:31.320920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:31.320920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:31.322410 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:31.322410 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:31.322410 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:31.322410 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 396/771] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:31.322645 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd1c40-0fc0-433f-bae4-331db2c0b416 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:31.322878 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd1c40-0fc0-433f-bae4-331db2c0b416 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:31.323142 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd1c40-0fc0-433f-bae4-331db2c0b416 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:31.324763 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 397/772] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:29:31.329570 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfd1c40-0fc0-433f-bae4-331db2c0b416 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:31.349812 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:31.363458 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db4b42b-de95-4d17-b39e-aaa83a350f87 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:31.401908 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc73f9f-3232-4294-944a-54be94ce41b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:31.402616 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc73f9f-3232-4294-944a-54be94ce41b5 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:31.403011 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc73f9f-3232-4294-944a-54be94ce41b5 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:31.403361 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc73f9f-3232-4294-944a-54be94ce41b5 None admin] PATH_INFO: `/v3/services/network` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:31.404113 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc73f9f-3232-4294-944a-54be94ce41b5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:31.406033 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc73f9f-3232-4294-944a-54be94ce41b5 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:31.416350 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-1fc73f9f-3232-4294-944a-54be94ce41b5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 22:29:31.416350 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:31.416350 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:29:31.416350 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:29:31.416350 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:29:31.416350 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:29:31.416350 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:29:31.416350 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:29:31.416350 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 22:29:31.416350 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:31.416350 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:29:31.416350 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:31.416350 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:31.416350 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:31.416350 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:31.416350 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:31.416350 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:31.417402 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:31.417402 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:31.417402 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:31.417402 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:31.417402 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:31.417402 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:31.417402 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:29:31.417402 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:29:31.417402 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:29:31.417402 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:29:31.417402 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:31.417402 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:31.417402 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:31.417402 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:31.417402 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:31.417402 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:31.418221 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:31.418221 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:31.418221 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:31.418221 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:31.418221 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:31.418221 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:31.418221 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:31.418221 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:31.418221 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:31.418221 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:31.418221 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:29:31.418221 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:29:31.418221 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 22:29:31.418221 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:31.418221 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 376/773] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:29:31 2022] GET /identity/v3/services/network => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:31.419606 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494c47d-1346-4a84-be38-2d8a9915a67b None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:31.419974 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494c47d-1346-4a84-be38-2d8a9915a67b None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:31.420634 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494c47d-1346-4a84-be38-2d8a9915a67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:31.442085 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-b494c47d-1346-4a84-be38-2d8a9915a67b None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:31.443492 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-b494c47d-1346-4a84-be38-2d8a9915a67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:31.448153 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfd1c40-0fc0-433f-bae4-331db2c0b416 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:31.485944 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57e428-2b8d-4cdd-8f8f-b968ea2e406d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:31.487521 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57e428-2b8d-4cdd-8f8f-b968ea2e406d None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:31.488148 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57e428-2b8d-4cdd-8f8f-b968ea2e406d None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:31.488606 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57e428-2b8d-4cdd-8f8f-b968ea2e406d None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:31.489572 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57e428-2b8d-4cdd-8f8f-b968ea2e406d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:31.490135 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57e428-2b8d-4cdd-8f8f-b968ea2e406d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:31.491805 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57e428-2b8d-4cdd-8f8f-b968ea2e406d None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:31.506708 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 398/774] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:29:31.507864 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57e428-2b8d-4cdd-8f8f-b968ea2e406d None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:31.529083 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 377/775] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:29:31.530186 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494c47d-1346-4a84-be38-2d8a9915a67b None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:31.567800 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:31.620419 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57e428-2b8d-4cdd-8f8f-b968ea2e406d None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:31.653233 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:31.660369 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf01e3a-7daf-40ec-8a49-ea940ed0cf7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:31.661303 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf01e3a-7daf-40ec-8a49-ea940ed0cf7a None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:31.661796 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf01e3a-7daf-40ec-8a49-ea940ed0cf7a None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:31.666109 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf01e3a-7daf-40ec-8a49-ea940ed0cf7a None admin] PATH_INFO: `/v3/services` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:31.666109 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf01e3a-7daf-40ec-8a49-ea940ed0cf7a None admin] RBAC: Adding query filter params (name=network) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:31.666109 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf01e3a-7daf-40ec-8a49-ea940ed0cf7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:31.666109 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf01e3a-7daf-40ec-8a49-ea940ed0cf7a None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:31.670347 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 399/776] 10.222.0.52 () {56 vars in 1112 bytes} [Mon Aug 29 22:29:31 2022] GET /identity/v3/services?name=network => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:31.671816 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c85118-d59d-4355-b709-0813c311b783 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:31.672324 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c85118-d59d-4355-b709-0813c311b783 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:31.672824 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c85118-d59d-4355-b709-0813c311b783 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:31.675132 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494c47d-1346-4a84-be38-2d8a9915a67b None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:31.675554 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 400/777] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:29:31.676497 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c85118-d59d-4355-b709-0813c311b783 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:31.715547 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e47659-e9dd-4760-a87d-9d1633da5222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:31.716219 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e47659-e9dd-4760-a87d-9d1633da5222 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:31.716503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e47659-e9dd-4760-a87d-9d1633da5222 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:31.716841 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e47659-e9dd-4760-a87d-9d1633da5222 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:31.724695 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e47659-e9dd-4760-a87d-9d1633da5222 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:31.726338 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e47659-e9dd-4760-a87d-9d1633da5222 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:31.738632 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-52e47659-e9dd-4760-a87d-9d1633da5222 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:31.738632 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:31.738632 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:31.738632 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:31.738632 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:31.738632 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:31.738632 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:31.738632 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:31.738632 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:31.738632 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:31.738632 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:31.738632 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:31.738632 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:29:31.738632 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:29:31.738632 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:29:31.739548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:29:31.739548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:31.739548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:31.739548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:31.739548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:31.739548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:31.739548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:31.739548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:31.739548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:31.739548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:31.739548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:31.739548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:31.739548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:31.739548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:31.739548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:31.739548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:31.739548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:31.740419 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:29:31.740419 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:29:31.740419 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:29:31.740419 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:29:31.740419 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:29:31.740419 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:29:31.740419 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:31.740419 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:31.740419 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 378/778] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:29:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:31.741687 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf801a-d27e-4648-961a-2c02bf78b36b None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:31.742400 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf801a-d27e-4648-961a-2c02bf78b36b None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:31.742400 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf801a-d27e-4648-961a-2c02bf78b36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:31.761462 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-abaf801a-d27e-4648-961a-2c02bf78b36b None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:31.762524 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-abaf801a-d27e-4648-961a-2c02bf78b36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:31.795281 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c85118-d59d-4355-b709-0813c311b783 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:31.812115 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:31.835465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b883f-ea8d-4529-a0c6-b8d9729f2cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:31.835865 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b883f-ea8d-4529-a0c6-b8d9729f2cd0 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:31.836148 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b883f-ea8d-4529-a0c6-b8d9729f2cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:31.836428 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b883f-ea8d-4529-a0c6-b8d9729f2cd0 None admin] PATH_INFO: `/v3/services` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:31.837107 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036b883f-ea8d-4529-a0c6-b8d9729f2cd0 None admin] RBAC: Adding query filter params (type=network) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:31.837412 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036b883f-ea8d-4529-a0c6-b8d9729f2cd0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:31.838770 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036b883f-ea8d-4529-a0c6-b8d9729f2cd0 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:31.841357 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 379/779] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:29:31.842604 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf801a-d27e-4648-961a-2c02bf78b36b None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:31.844809 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 401/780] 10.222.0.52 () {56 vars in 1112 bytes} [Mon Aug 29 22:29:31 2022] GET /identity/v3/services?type=network => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:31.846163 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b883f-ea8d-4529-a0c6-b8d9729f2cd0 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:31.921520 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:31.953897 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf801a-d27e-4648-961a-2c02bf78b36b None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:31.989262 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b883f-ea8d-4529-a0c6-b8d9729f2cd0 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:31.994444 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a4a22-6eef-43eb-b05b-d285f887542b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:31.995164 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a4a22-6eef-43eb-b05b-d285f887542b None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:31.995549 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a4a22-6eef-43eb-b05b-d285f887542b None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:31.995927 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a4a22-6eef-43eb-b05b-d285f887542b None admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:31.996774 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119a4a22-6eef-43eb-b05b-d285f887542b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:31.997194 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119a4a22-6eef-43eb-b05b-d285f887542b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:31.998729 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119a4a22-6eef-43eb-b05b-d285f887542b None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:32.011830 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 380/781] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Aug 29 22:29:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:32.012854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a4a22-6eef-43eb-b05b-d285f887542b None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:32.034880 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc438510-5708-4ef2-bbf5-30a1b811dad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:32.035715 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc438510-5708-4ef2-bbf5-30a1b811dad3 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:32.036109 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc438510-5708-4ef2-bbf5-30a1b811dad3 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:32.036477 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc438510-5708-4ef2-bbf5-30a1b811dad3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:32.048107 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc438510-5708-4ef2-bbf5-30a1b811dad3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:29:32.053946 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-bc438510-5708-4ef2-bbf5-30a1b811dad3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:29:32.053946 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:32.053946 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:32.053946 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:32.053946 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:32.053946 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:32.053946 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:32.053946 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:32.053946 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:32.053946 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:32.053946 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:32.053946 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:32.053946 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:29:32.053946 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:29:32.053946 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:29:32.053946 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:29:32.055150 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:29:32.055150 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:29:32.055150 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:29:32.055150 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:29:32.055150 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:32.055150 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:32.055150 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:32.055150 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:32.055150 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:32.055150 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:32.055150 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:32.055150 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:32.055150 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:32.055150 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:32.055150 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:32.055150 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:32.056207 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:32.056207 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:32.056207 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:32.056207 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:32.056207 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:29:32.056207 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:29:32.056207 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:29:32.056207 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:29:32.056207 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:29:32.056207 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:29:32.056207 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:29:32.056207 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:32.058135 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 402/782] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Aug 29 22:29:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:32.058975 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc438510-5708-4ef2-bbf5-30a1b811dad3 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:32.104370 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:32.150486 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a4a22-6eef-43eb-b05b-d285f887542b None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:32.174626 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc438510-5708-4ef2-bbf5-30a1b811dad3 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:32.184323 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:32.189363 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3038e23-2d40-436a-bea1-cef7f323d2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:32.190998 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3038e23-2d40-436a-bea1-cef7f323d2c9 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:32.190998 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3038e23-2d40-436a-bea1-cef7f323d2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:32.191453 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3038e23-2d40-436a-bea1-cef7f323d2c9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:32.192199 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3038e23-2d40-436a-bea1-cef7f323d2c9 None admin] RBAC: Adding query filter params (interface=public, service_id=e3dace5f780e4781a79c269a5d2992f6, region_id=RegionOne) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:32.192555 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3038e23-2d40-436a-bea1-cef7f323d2c9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:32.194009 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3038e23-2d40-436a-bea1-cef7f323d2c9 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:32.199333 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 381/783] 10.222.0.52 () {56 vars in 1251 bytes} [Mon Aug 29 22:29:32 2022] GET /identity/v3/endpoints?service_id=e3dace5f780e4781a79c269a5d2992f6&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:32.204433 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3038e23-2d40-436a-bea1-cef7f323d2c9 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:32.215739 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb6512-e027-415e-96a8-4e349f346f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:32.216429 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb6512-e027-415e-96a8-4e349f346f91 None admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:32.216712 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb6512-e027-415e-96a8-4e349f346f91 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:32.217014 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb6512-e027-415e-96a8-4e349f346f91 None admin] PATH_INFO: `/v3/projects/ee84c3340f2641ad963655a7a212ef96/users/bdbf961e02934e88af3a72d6218f32fa/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:32.218952 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afb6512-e027-415e-96a8-4e349f346f91 None admin] RBAC: Authorizing `identity:create_grant(project_id=ee84c3340f2641ad963655a7a212ef96, user_id=bdbf961e02934e88af3a72d6218f32fa, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:32.220391 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afb6512-e027-415e-96a8-4e349f346f91 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:32.293749 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 403/784] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Aug 29 22:29:32 2022] PUT /identity/v3/projects/ee84c3340f2641ad963655a7a212ef96/users/bdbf961e02934e88af3a72d6218f32fa/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:29:32.294566 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb6512-e027-415e-96a8-4e349f346f91 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:32.326345 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3038e23-2d40-436a-bea1-cef7f323d2c9 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:32.381975 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ce997-b0d2-4185-be94-6968154f9768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:32.382597 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ce997-b0d2-4185-be94-6968154f9768 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:32.382872 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ce997-b0d2-4185-be94-6968154f9768 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:32.383142 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ce997-b0d2-4185-be94-6968154f9768 None admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:32.383888 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97ce997-b0d2-4185-be94-6968154f9768 None admin] RBAC: Adding query filter params (name=service) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:32.384108 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97ce997-b0d2-4185-be94-6968154f9768 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:32.385915 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97ce997-b0d2-4185-be94-6968154f9768 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:32.395453 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 382/785] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Aug 29 22:29:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:32.396391 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ce997-b0d2-4185-be94-6968154f9768 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:32.435981 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:32.443013 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb6512-e027-415e-96a8-4e349f346f91 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:32.469841 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:32.500997 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64625fb3-56a0-4aa7-9a32-b4ce44044d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:32.501629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64625fb3-56a0-4aa7-9a32-b4ce44044d8a None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:32.501981 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64625fb3-56a0-4aa7-9a32-b4ce44044d8a None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:32.502756 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64625fb3-56a0-4aa7-9a32-b4ce44044d8a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:32.514085 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64625fb3-56a0-4aa7-9a32-b4ce44044d8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:29:32.519504 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-64625fb3-56a0-4aa7-9a32-b4ce44044d8a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:29:32.519504 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:32.519504 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:32.519504 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:32.519504 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:32.519504 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:32.519504 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:32.519504 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:32.519504 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:32.519504 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:32.519504 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:32.519504 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:32.519504 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:29:32.519504 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:29:32.519504 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:29:32.519504 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:29:32.520624 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:29:32.520624 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:29:32.520624 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:29:32.520624 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:29:32.520624 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:32.520624 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:32.520624 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:32.520624 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:32.520624 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:32.520624 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:32.520624 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:32.520624 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:32.520624 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:32.520624 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:32.520624 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:32.520624 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:32.521722 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:32.521722 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:32.521722 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:32.521722 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:32.521722 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:29:32.521722 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:29:32.521722 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:29:32.521722 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:29:32.521722 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:29:32.521722 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:29:32.521722 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:29:32.521722 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:32.524423 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 404/786] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Aug 29 22:29:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:32.524844 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ce997-b0d2-4185-be94-6968154f9768 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:32.529960 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64625fb3-56a0-4aa7-9a32-b4ce44044d8a None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:32.568340 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c0b6d9-170f-49d0-8877-4b7b17887d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:32.569298 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0b6d9-170f-49d0-8877-4b7b17887d42 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:32.569771 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0b6d9-170f-49d0-8877-4b7b17887d42 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:32.570170 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0b6d9-170f-49d0-8877-4b7b17887d42 None admin] PATH_INFO: `/v3/services` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:32.570932 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c0b6d9-170f-49d0-8877-4b7b17887d42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:32.572549 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c0b6d9-170f-49d0-8877-4b7b17887d42 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:32.578850 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 383/787] 10.222.0.52 () {56 vars in 1087 bytes} [Mon Aug 29 22:29:32 2022] GET /identity/v3/services => generated 772 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:32.585931 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c0b6d9-170f-49d0-8877-4b7b17887d42 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:32.679442 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64625fb3-56a0-4aa7-9a32-b4ce44044d8a None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:32.757041 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd9edf-8780-463b-96bc-df9c51f4c034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:32.757688 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd9edf-8780-463b-96bc-df9c51f4c034 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:32.757963 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd9edf-8780-463b-96bc-df9c51f4c034 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:32.758231 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd9edf-8780-463b-96bc-df9c51f4c034 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:32.771681 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bd9edf-8780-463b-96bc-df9c51f4c034 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:32.773305 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bd9edf-8780-463b-96bc-df9c51f4c034 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:32.792592 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-99bd9edf-8780-463b-96bc-df9c51f4c034 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:32.792592 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:32.792592 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:32.792592 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:32.792592 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:32.792592 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:32.792592 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:32.792592 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:32.792592 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:32.792592 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:32.792592 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:32.792592 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:32.792592 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:32.792592 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:32.792592 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:32.792592 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:32.793631 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:32.793631 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:32.793631 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:32.793631 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:32.793631 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:32.793631 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:32.793631 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:32.793631 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:32.793631 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:32.793631 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:32.793631 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:32.793631 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:32.793631 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:32.793631 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:32.793631 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:32.793631 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:32.793631 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:32.794523 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:32.794523 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:32.794523 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:32.794523 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 405/788] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:32.801607 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd9edf-8780-463b-96bc-df9c51f4c034 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:32.807823 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c0b6d9-170f-49d0-8877-4b7b17887d42 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:32.820245 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:32.839948 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:32.855207 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5f0dda-758f-4d4a-8f79-3c323a8810dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:32.855868 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f0dda-758f-4d4a-8f79-3c323a8810dd None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:32.856152 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f0dda-758f-4d4a-8f79-3c323a8810dd None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:32.856423 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f0dda-758f-4d4a-8f79-3c323a8810dd None admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:32.857520 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5f0dda-758f-4d4a-8f79-3c323a8810dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:32.857827 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5f0dda-758f-4d4a-8f79-3c323a8810dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:32.859659 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5f0dda-758f-4d4a-8f79-3c323a8810dd None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:32.870352 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 384/789] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:29:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:32.876116 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5f0dda-758f-4d4a-8f79-3c323a8810dd None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:32.989673 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5f0dda-758f-4d4a-8f79-3c323a8810dd None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:33.008073 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd9edf-8780-463b-96bc-df9c51f4c034 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:33.027451 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec8415a-d059-476e-b3f8-452ba3011309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:33.028083 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec8415a-d059-476e-b3f8-452ba3011309 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:33.028363 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec8415a-d059-476e-b3f8-452ba3011309 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:33.028635 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec8415a-d059-476e-b3f8-452ba3011309 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:33.040956 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec8415a-d059-476e-b3f8-452ba3011309 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:33.042491 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec8415a-d059-476e-b3f8-452ba3011309 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:33.058472 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-cec8415a-d059-476e-b3f8-452ba3011309 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:33.058472 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:33.058472 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:33.058472 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:33.058472 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:33.058472 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:33.058472 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:33.058472 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:33.058472 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:33.058472 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:33.058472 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:33.058472 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:33.058472 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:33.058472 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:33.058472 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:33.058472 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:33.059554 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:33.059554 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:33.059554 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:33.059554 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:33.059554 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:33.059554 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:33.059554 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:33.059554 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:33.059554 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:33.059554 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:33.059554 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:33.059554 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:33.059554 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:33.059554 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:33.059554 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:33.059554 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:33.059554 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:33.060494 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:33.060494 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:33.060494 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:33.060494 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 385/790] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:33.065965 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec8415a-d059-476e-b3f8-452ba3011309 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:33.082863 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70bafe-d306-4a84-a9ba-0e6d738a2699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:33.083479 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70bafe-d306-4a84-a9ba-0e6d738a2699 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:33.083754 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70bafe-d306-4a84-a9ba-0e6d738a2699 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:33.084020 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70bafe-d306-4a84-a9ba-0e6d738a2699 None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:33.084662 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef70bafe-d306-4a84-a9ba-0e6d738a2699 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:33.084989 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef70bafe-d306-4a84-a9ba-0e6d738a2699 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:33.086520 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef70bafe-d306-4a84-a9ba-0e6d738a2699 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:33.105161 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:33.114656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 406/791] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:33.120340 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70bafe-d306-4a84-a9ba-0e6d738a2699 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:33.190766 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec8415a-d059-476e-b3f8-452ba3011309 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:33.240948 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b88d0-ce92-480f-9e44-3ad11e13e2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:33.241556 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b88d0-ce92-480f-9e44-3ad11e13e2a8 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:33.241835 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b88d0-ce92-480f-9e44-3ad11e13e2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:33.242108 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b88d0-ce92-480f-9e44-3ad11e13e2a8 None admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:33.243019 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b88d0-ce92-480f-9e44-3ad11e13e2a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:33.243326 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b88d0-ce92-480f-9e44-3ad11e13e2a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:33.245508 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b88d0-ce92-480f-9e44-3ad11e13e2a8 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:33.247193 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:33.271042 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 386/792] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:33.272320 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b427a-62ed-46c1-89ba-d1e5000c8084 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:33.272600 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b427a-62ed-46c1-89ba-d1e5000c8084 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:33.272928 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b427a-62ed-46c1-89ba-d1e5000c8084 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:33.278293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 387/793] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:29:33.279076 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46b427a-62ed-46c1-89ba-d1e5000c8084 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:33.285935 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70bafe-d306-4a84-a9ba-0e6d738a2699 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:33.335676 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18333af-83b1-4119-8429-d14e9ade68ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:33.336919 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18333af-83b1-4119-8429-d14e9ade68ae None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:33.337381 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18333af-83b1-4119-8429-d14e9ade68ae None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:33.338207 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18333af-83b1-4119-8429-d14e9ade68ae None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:33.348782 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18333af-83b1-4119-8429-d14e9ade68ae None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:33.351596 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18333af-83b1-4119-8429-d14e9ade68ae None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:33.368061 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-f18333af-83b1-4119-8429-d14e9ade68ae None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 22:29:33.368061 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:33.368061 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:33.368061 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:33.368061 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:33.368061 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:33.368061 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:33.368061 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:33.368061 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:33.368061 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:33.368061 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:33.368061 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:33.368061 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:29:33.368061 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:29:33.368061 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:29:33.368061 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:29:33.369108 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:33.369108 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:33.369108 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:29:33.369108 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:33.369108 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:29:33.369108 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:33.369108 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:33.369108 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:33.369108 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:33.369108 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:33.369108 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:33.369108 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:33.369108 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:33.369108 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:33.369108 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:33.369108 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:33.369108 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:33.369108 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:33.370043 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:33.370043 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:33.370043 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:29:33.370043 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:29:33.370043 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:29:33.370043 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:29:33.370043 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:29:33.370043 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:29:33.370043 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 22:29:33.370043 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:33.372785 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 407/794] 10.222.0.52 () {56 vars in 1117 bytes} [Mon Aug 29 22:29:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:29:33.374176 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc1e2e-4891-45ac-8579-60d5851ed281 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:33.374589 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc1e2e-4891-45ac-8579-60d5851ed281 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:33.375010 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc1e2e-4891-45ac-8579-60d5851ed281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:33.394825 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:33.401848 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46b427a-62ed-46c1-89ba-d1e5000c8084 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:33.407795 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-11dc1e2e-4891-45ac-8579-60d5851ed281 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:33.409833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-11dc1e2e-4891-45ac-8579-60d5851ed281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:33.440145 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f5d5b2-b78d-4cfe-9234-136453901d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:33.441188 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5d5b2-b78d-4cfe-9234-136453901d9c None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:33.441623 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5d5b2-b78d-4cfe-9234-136453901d9c None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:33.441999 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5d5b2-b78d-4cfe-9234-136453901d9c None admin] PATH_INFO: `/v3/users/swift` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:33.450705 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f5d5b2-b78d-4cfe-9234-136453901d9c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:33.452406 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f5d5b2-b78d-4cfe-9234-136453901d9c None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:33.466518 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-81f5d5b2-b78d-4cfe-9234-136453901d9c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 22:29:33.466518 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:33.466518 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:33.466518 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:33.466518 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:33.466518 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:33.466518 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:33.466518 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:33.466518 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:33.466518 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:33.466518 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:33.466518 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:33.466518 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:29:33.466518 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:29:33.466518 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:29:33.466518 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:29:33.467693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:33.467693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:33.467693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:29:33.467693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:33.467693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:29:33.467693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:33.467693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:33.467693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:33.467693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:33.467693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:33.467693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:33.467693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:33.467693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:33.467693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:33.467693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:33.467693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:33.467693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:33.467693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:33.468768 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:33.468768 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:33.468768 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:29:33.468768 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:29:33.468768 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:29:33.468768 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:29:33.468768 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:29:33.468768 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:29:33.468768 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 22:29:33.468768 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:33.473152 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 388/795] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Aug 29 22:29:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:33.474046 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f5d5b2-b78d-4cfe-9234-136453901d9c None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:33.501775 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 408/796] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:29:33.503458 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dc1e2e-4891-45ac-8579-60d5851ed281 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:33.581189 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:33.615883 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f5d5b2-b78d-4cfe-9234-136453901d9c None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:33.627402 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dc1e2e-4891-45ac-8579-60d5851ed281 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:33.655996 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b56260-86bf-42f0-84ad-95959cbc74bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:33.656908 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b56260-86bf-42f0-84ad-95959cbc74bb None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:33.657294 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b56260-86bf-42f0-84ad-95959cbc74bb None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:33.657695 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b56260-86bf-42f0-84ad-95959cbc74bb None admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:33.659217 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b56260-86bf-42f0-84ad-95959cbc74bb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:33.659674 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b56260-86bf-42f0-84ad-95959cbc74bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:33.661932 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b56260-86bf-42f0-84ad-95959cbc74bb None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:33.668075 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a487e7-764d-4786-9d12-d3f5080a1d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:33.668925 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a487e7-764d-4786-9d12-d3f5080a1d1b None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:33.669379 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a487e7-764d-4786-9d12-d3f5080a1d1b None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:33.669808 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a487e7-764d-4786-9d12-d3f5080a1d1b None admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:33.670691 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a487e7-764d-4786-9d12-d3f5080a1d1b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:33.671143 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a487e7-764d-4786-9d12-d3f5080a1d1b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:33.672697 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a487e7-764d-4786-9d12-d3f5080a1d1b None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:33.685808 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 389/797] 10.222.0.52 () {56 vars in 1149 bytes} [Mon Aug 29 22:29:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:33.695611 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b56260-86bf-42f0-84ad-95959cbc74bb None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:33.700614 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 409/798] 10.222.0.52 () {56 vars in 1135 bytes} [Mon Aug 29 22:29:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:33.706892 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a487e7-764d-4786-9d12-d3f5080a1d1b None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:33.729205 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:33.812453 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a487e7-764d-4786-9d12-d3f5080a1d1b None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:33.836428 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:33.851324 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5012050-5d97-4d85-84b4-92a47e9b7301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:33.852456 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5012050-5d97-4d85-84b4-92a47e9b7301 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:33.852988 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5012050-5d97-4d85-84b4-92a47e9b7301 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:33.853468 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5012050-5d97-4d85-84b4-92a47e9b7301 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:33.866244 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5012050-5d97-4d85-84b4-92a47e9b7301 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:33.868060 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5012050-5d97-4d85-84b4-92a47e9b7301 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:33.884022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-e5012050-5d97-4d85-84b4-92a47e9b7301 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:33.884022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:33.884022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:33.884022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:33.884022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:33.884022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:33.884022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:33.884022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:33.884022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:33.884022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:33.884022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:33.884022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:33.884022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:33.884022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:33.884022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:33.884022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:33.885033 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:33.885033 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:33.885033 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:33.885033 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:33.885033 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:33.885033 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:33.885033 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:33.885033 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:33.885033 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:33.885033 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:33.885033 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:33.885033 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:33.885033 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:33.885033 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:33.885033 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:33.885033 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:33.885033 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:33.885951 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:33.885951 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:33.885951 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:33.886763 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b56260-86bf-42f0-84ad-95959cbc74bb None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:33.888629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 410/799] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:29:33.895351 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5012050-5d97-4d85-84b4-92a47e9b7301 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:33.962892 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff31de5e-6d8d-4f2e-acdb-ed168ba69c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:33.963907 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff31de5e-6d8d-4f2e-acdb-ed168ba69c61 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:33.964427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff31de5e-6d8d-4f2e-acdb-ed168ba69c61 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:33.964955 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff31de5e-6d8d-4f2e-acdb-ed168ba69c61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:33.985502 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff31de5e-6d8d-4f2e-acdb-ed168ba69c61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:33.987394 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff31de5e-6d8d-4f2e-acdb-ed168ba69c61 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:34.018364 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-ff31de5e-6d8d-4f2e-acdb-ed168ba69c61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:34.018364 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:34.018364 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:34.018364 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:34.018364 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:34.018364 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:34.018364 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:34.018364 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:34.018364 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:34.018364 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:34.018364 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:34.018364 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:34.018364 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:34.018364 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:34.018364 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:34.018364 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:34.019391 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:34.019391 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:34.019391 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:34.019391 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:34.019391 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:34.019391 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:34.019391 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:34.019391 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:34.019391 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:34.019391 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:34.019391 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:34.019391 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:34.019391 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:34.019391 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:34.019391 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:34.019391 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:34.019391 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:34.020300 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:34.020300 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:34.020300 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:34.034693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 390/800] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:29:34.049683 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff31de5e-6d8d-4f2e-acdb-ed168ba69c61 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:34.073929 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5012050-5d97-4d85-84b4-92a47e9b7301 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:34.113857 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92f4ca-abbf-4538-ad9b-0387d96e90f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:34.114474 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92f4ca-abbf-4538-ad9b-0387d96e90f3 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:34.114758 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92f4ca-abbf-4538-ad9b-0387d96e90f3 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:34.115034 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92f4ca-abbf-4538-ad9b-0387d96e90f3 None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:34.115694 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e92f4ca-abbf-4538-ad9b-0387d96e90f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:34.115999 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e92f4ca-abbf-4538-ad9b-0387d96e90f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:34.117421 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e92f4ca-abbf-4538-ad9b-0387d96e90f3 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:34.131055 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 411/801] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:29:34.131915 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92f4ca-abbf-4538-ad9b-0387d96e90f3 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:34.157108 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:34.167180 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff31de5e-6d8d-4f2e-acdb-ed168ba69c61 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:34.173169 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:34.206976 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee16171f-20c6-4533-87df-dab8ae25caae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:34.208028 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16171f-20c6-4533-87df-dab8ae25caae None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:34.208559 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16171f-20c6-4533-87df-dab8ae25caae None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:34.209098 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16171f-20c6-4533-87df-dab8ae25caae None admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:34.209971 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee16171f-20c6-4533-87df-dab8ae25caae None admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:34.210495 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee16171f-20c6-4533-87df-dab8ae25caae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:34.212080 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee16171f-20c6-4533-87df-dab8ae25caae None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:34.226423 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 391/802] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:29:34.227790 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee16171f-20c6-4533-87df-dab8ae25caae None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:34.246740 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92f4ca-abbf-4538-ad9b-0387d96e90f3 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:34.282899 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9627db5-8e09-4b2d-be5a-d6907d6e6f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:34.283947 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9627db5-8e09-4b2d-be5a-d6907d6e6f4c None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:34.284228 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9627db5-8e09-4b2d-be5a-d6907d6e6f4c None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:34.284496 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9627db5-8e09-4b2d-be5a-d6907d6e6f4c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:34.285123 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9627db5-8e09-4b2d-be5a-d6907d6e6f4c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:34.286496 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9627db5-8e09-4b2d-be5a-d6907d6e6f4c None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:34.295764 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-e9627db5-8e09-4b2d-be5a-d6907d6e6f4c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 22:29:34.295764 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:34.295764 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:29:34.295764 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:29:34.295764 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:29:34.295764 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:29:34.295764 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:29:34.295764 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:29:34.295764 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 22:29:34.295764 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:34.295764 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:29:34.295764 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:34.295764 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:34.295764 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:34.295764 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:34.295764 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:34.295764 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:34.296874 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:34.296874 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:34.296874 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:34.296874 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:34.296874 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:34.296874 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:34.296874 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:29:34.296874 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:29:34.296874 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:29:34.296874 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:29:34.296874 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:34.296874 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:34.296874 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:34.296874 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:34.296874 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:34.296874 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:34.297721 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:34.297721 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:34.297721 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:34.297721 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:34.297721 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:34.297721 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:34.297721 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:34.297721 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:34.297721 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:34.297721 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:34.297721 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:29:34.297721 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:29:34.297721 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 22:29:34.297721 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:34.302770 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 412/803] 10.222.0.52 () {56 vars in 1129 bytes} [Mon Aug 29 22:29:34 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:34.303919 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9627db5-8e09-4b2d-be5a-d6907d6e6f4c None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:34.360837 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee16171f-20c6-4533-87df-dab8ae25caae None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:34.394357 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:34.405431 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7136ce-5c1a-4807-ba51-c09d0b5a4629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:34.406851 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7136ce-5c1a-4807-ba51-c09d0b5a4629 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:34.407555 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7136ce-5c1a-4807-ba51-c09d0b5a4629 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:34.408077 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7136ce-5c1a-4807-ba51-c09d0b5a4629 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:34.412050 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:34.416817 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7136ce-5c1a-4807-ba51-c09d0b5a4629 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:34.421138 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9627db5-8e09-4b2d-be5a-d6907d6e6f4c None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:34.421669 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7136ce-5c1a-4807-ba51-c09d0b5a4629 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:34.438630 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-8c7136ce-5c1a-4807-ba51-c09d0b5a4629 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:34.438630 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:34.438630 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:34.438630 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:34.438630 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:34.438630 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:34.438630 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:34.438630 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:34.438630 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:34.438630 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:34.438630 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:34.438630 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:34.438630 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:29:34.438630 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:29:34.438630 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:29:34.439923 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:29:34.439923 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:34.439923 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:34.439923 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:34.439923 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:34.439923 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:34.439923 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:34.439923 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:34.439923 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:34.439923 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:34.439923 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:34.439923 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:34.439923 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:34.439923 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:34.439923 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:34.439923 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:34.439923 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:34.441099 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:29:34.441099 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:29:34.441099 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:29:34.441099 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:29:34.441099 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:29:34.441099 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:29:34.441099 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:34.441099 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:34.443206 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 392/804] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:29:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:34.444265 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7136ce-5c1a-4807-ba51-c09d0b5a4629 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:34.462158 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e563efe4-c96c-4ea5-b658-cb01a678b351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:34.462848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e563efe4-c96c-4ea5-b658-cb01a678b351 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:34.463339 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e563efe4-c96c-4ea5-b658-cb01a678b351 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:34.463448 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e563efe4-c96c-4ea5-b658-cb01a678b351 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:34.470668 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e563efe4-c96c-4ea5-b658-cb01a678b351 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:34.472946 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e563efe4-c96c-4ea5-b658-cb01a678b351 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:34.484957 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-e563efe4-c96c-4ea5-b658-cb01a678b351 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:34.484957 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:34.484957 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:34.484957 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:34.484957 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:34.484957 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:34.484957 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:34.484957 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:34.484957 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:34.484957 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:34.484957 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:34.484957 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:34.484957 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:29:34.484957 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:29:34.484957 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:29:34.486060 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:29:34.486060 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:34.486060 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:34.486060 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:34.486060 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:34.486060 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:34.486060 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:34.486060 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:34.486060 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:34.486060 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:34.486060 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:34.486060 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:34.486060 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:34.486060 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:34.486060 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:34.486060 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:34.486060 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:34.487453 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:29:34.487453 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:29:34.487453 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:29:34.487453 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:29:34.487453 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:29:34.487453 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:29:34.487453 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:34.487453 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:34.487453 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 413/805] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:29:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:34.487453 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e563efe4-c96c-4ea5-b658-cb01a678b351 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:34.563874 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7136ce-5c1a-4807-ba51-c09d0b5a4629 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:34.618359 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a51ef-8464-4a81-8ffa-7ede6dbe812d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:34.618771 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e563efe4-c96c-4ea5-b658-cb01a678b351 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:34.618881 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a51ef-8464-4a81-8ffa-7ede6dbe812d None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:34.619211 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a51ef-8464-4a81-8ffa-7ede6dbe812d None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:34.619490 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a51ef-8464-4a81-8ffa-7ede6dbe812d None admin] PATH_INFO: `/v3/services` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:34.620172 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746a51ef-8464-4a81-8ffa-7ede6dbe812d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:34.620478 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746a51ef-8464-4a81-8ffa-7ede6dbe812d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:34.623088 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746a51ef-8464-4a81-8ffa-7ede6dbe812d None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:34.634353 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 393/806] 10.222.0.52 () {56 vars in 1124 bytes} [Mon Aug 29 22:29:34 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:34.636107 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50b901-6916-4da7-9cc6-bd5a4f254b49 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:34.636401 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50b901-6916-4da7-9cc6-bd5a4f254b49 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:34.636981 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50b901-6916-4da7-9cc6-bd5a4f254b49 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:34.638796 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 394/807] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:29:34.639613 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50b901-6916-4da7-9cc6-bd5a4f254b49 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:34.670386 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32538e0-6a68-48d1-b6dc-7e3cad5ffbe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:34.671457 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32538e0-6a68-48d1-b6dc-7e3cad5ffbe2 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:34.671743 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32538e0-6a68-48d1-b6dc-7e3cad5ffbe2 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:34.672191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32538e0-6a68-48d1-b6dc-7e3cad5ffbe2 None admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:34.673062 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32538e0-6a68-48d1-b6dc-7e3cad5ffbe2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:34.673377 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32538e0-6a68-48d1-b6dc-7e3cad5ffbe2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:34.675400 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32538e0-6a68-48d1-b6dc-7e3cad5ffbe2 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:34.692431 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 414/808] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Aug 29 22:29:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:34.693340 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32538e0-6a68-48d1-b6dc-7e3cad5ffbe2 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:34.719125 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:34.719319 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:34.781076 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50b901-6916-4da7-9cc6-bd5a4f254b49 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:34.831816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd6154-08a0-445f-9f2f-c74e40b4dab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:34.832150 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32538e0-6a68-48d1-b6dc-7e3cad5ffbe2 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:34.833301 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd6154-08a0-445f-9f2f-c74e40b4dab8 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:34.833937 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd6154-08a0-445f-9f2f-c74e40b4dab8 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:34.834544 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd6154-08a0-445f-9f2f-c74e40b4dab8 None admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:34.835876 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbd6154-08a0-445f-9f2f-c74e40b4dab8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:34.836222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbd6154-08a0-445f-9f2f-c74e40b4dab8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:34.838770 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbd6154-08a0-445f-9f2f-c74e40b4dab8 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:34.858810 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 395/809] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Aug 29 22:29:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:34.860537 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d30ba-9af2-4f79-a880-19a79907f6c0 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:34.861070 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d30ba-9af2-4f79-a880-19a79907f6c0 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:34.861600 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d30ba-9af2-4f79-a880-19a79907f6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:34.886401 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-d96d30ba-9af2-4f79-a880-19a79907f6c0 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:34.888112 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-d96d30ba-9af2-4f79-a880-19a79907f6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:34.891757 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d007fe-1ecf-4ff3-8313-0e910176d8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:34.892469 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d007fe-1ecf-4ff3-8313-0e910176d8a2 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:34.893541 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d007fe-1ecf-4ff3-8313-0e910176d8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:34.893910 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d007fe-1ecf-4ff3-8313-0e910176d8a2 None admin] PATH_INFO: `/v3/services` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:34.894549 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d007fe-1ecf-4ff3-8313-0e910176d8a2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:34.894856 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d007fe-1ecf-4ff3-8313-0e910176d8a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:34.896220 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d007fe-1ecf-4ff3-8313-0e910176d8a2 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:34.901605 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 415/810] 10.222.0.52 () {56 vars in 1124 bytes} [Mon Aug 29 22:29:34 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:34.902372 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d007fe-1ecf-4ff3-8313-0e910176d8a2 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:34.980543 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 396/811] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:29:34.981654 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b5f5b4-70dc-4f7a-864d-590106a26761 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:34.981938 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b5f5b4-70dc-4f7a-864d-590106a26761 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:34.982210 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b5f5b4-70dc-4f7a-864d-590106a26761 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:34.983431 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 397/812] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:29:34.984090 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b5f5b4-70dc-4f7a-864d-590106a26761 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:34.997150 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:35.031414 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d007fe-1ecf-4ff3-8313-0e910176d8a2 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:35.038106 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:35.091637 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15982bef-b071-4a37-b3a5-fe14bff6ca61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:35.092377 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15982bef-b071-4a37-b3a5-fe14bff6ca61 None admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:35.092637 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15982bef-b071-4a37-b3a5-fe14bff6ca61 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:35.092940 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15982bef-b071-4a37-b3a5-fe14bff6ca61 None admin] PATH_INFO: `/v3/projects/ee84c3340f2641ad963655a7a212ef96/users/57c175ac90e545358746176972ff39a3/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:35.094955 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15982bef-b071-4a37-b3a5-fe14bff6ca61 None admin] RBAC: Authorizing `identity:create_grant(project_id=ee84c3340f2641ad963655a7a212ef96, user_id=57c175ac90e545358746176972ff39a3, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:35.096418 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15982bef-b071-4a37-b3a5-fe14bff6ca61 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:35.118967 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b5f5b4-70dc-4f7a-864d-590106a26761 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:35.169161 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 416/813] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Aug 29 22:29:34 2022] PUT /identity/v3/projects/ee84c3340f2641ad963655a7a212ef96/users/57c175ac90e545358746176972ff39a3/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:29:35.170228 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15982bef-b071-4a37-b3a5-fe14bff6ca61 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:35.185046 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b69fa-8e05-4cb0-a29c-144ddb917b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:35.186472 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b69fa-8e05-4cb0-a29c-144ddb917b3d None admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:35.186810 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b69fa-8e05-4cb0-a29c-144ddb917b3d None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:35.187092 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b69fa-8e05-4cb0-a29c-144ddb917b3d None admin] PATH_INFO: `/v3/projects/ee84c3340f2641ad963655a7a212ef96/users/f77dfe78b1a8405084c06d9acee96511/roles/db83c2f8e21945c3ab9642c620d29a23` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:35.219596 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b69fa-8e05-4cb0-a29c-144ddb917b3d None admin] RBAC: Authorizing `identity:create_grant(project_id=ee84c3340f2641ad963655a7a212ef96, user_id=f77dfe78b1a8405084c06d9acee96511, role_id=db83c2f8e21945c3ab9642c620d29a23)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:35.223633 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b69fa-8e05-4cb0-a29c-144ddb917b3d None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:35.300612 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 398/814] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Aug 29 22:29:34 2022] PUT /identity/v3/projects/ee84c3340f2641ad963655a7a212ef96/users/f77dfe78b1a8405084c06d9acee96511/roles/db83c2f8e21945c3ab9642c620d29a23 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:29:35.304529 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada49c72-b392-4815-951e-5553b2e28480 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:35.305364 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada49c72-b392-4815-951e-5553b2e28480 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:35.306990 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada49c72-b392-4815-951e-5553b2e28480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:35.332661 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-ada49c72-b392-4815-951e-5553b2e28480 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:35.334794 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-ada49c72-b392-4815-951e-5553b2e28480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:35.335204 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15982bef-b071-4a37-b3a5-fe14bff6ca61 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:35.399364 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2321099c-3b35-464b-8bf0-896c7b1679e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:35.399999 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2321099c-3b35-464b-8bf0-896c7b1679e6 None admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:35.400267 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2321099c-3b35-464b-8bf0-896c7b1679e6 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:35.401862 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2321099c-3b35-464b-8bf0-896c7b1679e6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:35.401862 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2321099c-3b35-464b-8bf0-896c7b1679e6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:35.405944 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2321099c-3b35-464b-8bf0-896c7b1679e6 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:35.422158 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:35.450586 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 399/815] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:29:35.451632 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada49c72-b392-4815-951e-5553b2e28480 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:35.484000 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:35.555269 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 417/816] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:29:35 2022] POST /identity/v3/endpoints => generated 346 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:29:35.612783 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada49c72-b392-4815-951e-5553b2e28480 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:35.694573 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9ac1e-7755-4857-88f9-ee5dfc2ada7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:35.695514 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9ac1e-7755-4857-88f9-ee5dfc2ada7e None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:35.696022 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9ac1e-7755-4857-88f9-ee5dfc2ada7e None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:35.696505 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9ac1e-7755-4857-88f9-ee5dfc2ada7e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:35.722265 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f9ac1e-7755-4857-88f9-ee5dfc2ada7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:29:35.727476 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-75f9ac1e-7755-4857-88f9-ee5dfc2ada7e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:29:35.727476 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:35.727476 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:35.727476 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:35.727476 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:35.727476 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:35.727476 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:35.727476 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:35.727476 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:35.727476 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:35.727476 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:35.727476 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:35.727476 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:29:35.727476 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:29:35.727476 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:29:35.727476 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:29:35.728430 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:29:35.728430 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:29:35.728430 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:29:35.728430 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:29:35.728430 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:35.728430 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:35.728430 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:35.728430 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:35.728430 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:35.728430 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:35.728430 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:35.728430 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:35.728430 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:35.728430 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:35.728430 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:35.728430 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:35.729311 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:35.729311 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:35.729311 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:35.729311 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:35.729311 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:29:35.729311 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:29:35.729311 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:29:35.729311 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:29:35.729311 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:29:35.729311 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:29:35.729311 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:29:35.729311 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:35.732861 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 400/817] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Aug 29 22:29:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:35.737910 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2321099c-3b35-464b-8bf0-896c7b1679e6 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:35.776323 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9ac1e-7755-4857-88f9-ee5dfc2ada7e None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:35.906479 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:35.916483 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2321099c-3b35-464b-8bf0-896c7b1679e6 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:35.941941 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9ac1e-7755-4857-88f9-ee5dfc2ada7e None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:35.969392 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e85594-734a-4028-87e1-f7e39a66a13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:35.969997 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e85594-734a-4028-87e1-f7e39a66a13f None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:35.970435 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e85594-734a-4028-87e1-f7e39a66a13f None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:35.970765 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e85594-734a-4028-87e1-f7e39a66a13f None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:35.971650 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e85594-734a-4028-87e1-f7e39a66a13f None admin] RBAC: Adding query filter params (name=admin) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:35.972111 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e85594-734a-4028-87e1-f7e39a66a13f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:35.973907 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e85594-734a-4028-87e1-f7e39a66a13f None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:35.985784 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 418/818] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:29:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:35.991750 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e85594-734a-4028-87e1-f7e39a66a13f None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:36.008118 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a9ef3-cd82-43c9-9308-d0030fb38964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:36.009269 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a9ef3-cd82-43c9-9308-d0030fb38964 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:36.009559 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a9ef3-cd82-43c9-9308-d0030fb38964 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:36.009839 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a9ef3-cd82-43c9-9308-d0030fb38964 None admin] PATH_INFO: `/v3/services/network` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:36.010746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634a9ef3-cd82-43c9-9308-d0030fb38964 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:36.013491 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634a9ef3-cd82-43c9-9308-d0030fb38964 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:36.033413 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-634a9ef3-cd82-43c9-9308-d0030fb38964 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 22:29:36.033413 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:36.033413 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:29:36.033413 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:29:36.033413 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:29:36.033413 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:29:36.033413 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:29:36.033413 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:29:36.033413 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 22:29:36.033413 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:36.033413 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:29:36.033413 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:36.033413 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:36.033413 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:36.033413 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:36.033413 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:36.033413 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:36.034585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:36.034585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:36.034585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:36.034585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:36.034585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:36.034585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:36.034585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:29:36.034585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:29:36.034585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:29:36.034585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:29:36.034585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:36.034585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:36.034585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:36.034585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:36.034585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:36.034585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:36.035409 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:36.035409 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:36.035409 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:36.035409 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:36.035409 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:36.035409 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:36.035409 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:36.035409 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:36.035409 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:36.035409 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:36.035409 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:29:36.035409 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:29:36.035409 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 22:29:36.035409 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:36.036345 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 401/819] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:29:35 2022] GET /identity/v3/services/network => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:36.042071 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a9ef3-cd82-43c9-9308-d0030fb38964 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:36.082809 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:36.188859 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a9ef3-cd82-43c9-9308-d0030fb38964 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:36.203048 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e85594-734a-4028-87e1-f7e39a66a13f None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:36.254937 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b1466e-f73b-42f1-baf1-b102e2c96731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:36.255474 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b1466e-f73b-42f1-baf1-b102e2c96731 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:36.255759 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b1466e-f73b-42f1-baf1-b102e2c96731 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:36.256033 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b1466e-f73b-42f1-baf1-b102e2c96731 None admin] PATH_INFO: `/v3/services` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:36.256678 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b1466e-f73b-42f1-baf1-b102e2c96731 None admin] RBAC: Adding query filter params (name=network) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:36.260361 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b1466e-f73b-42f1-baf1-b102e2c96731 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:36.262337 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b1466e-f73b-42f1-baf1-b102e2c96731 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:36.268459 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 402/820] 10.222.0.52 () {56 vars in 1112 bytes} [Mon Aug 29 22:29:36 2022] GET /identity/v3/services?name=network => generated 124 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:29:36.275386 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b1466e-f73b-42f1-baf1-b102e2c96731 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:36.286758 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:36.298379 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065329ce-350e-4d3e-9aaf-1b0ec97074f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:36.299097 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065329ce-350e-4d3e-9aaf-1b0ec97074f7 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:36.299380 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065329ce-350e-4d3e-9aaf-1b0ec97074f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:36.299674 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065329ce-350e-4d3e-9aaf-1b0ec97074f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:36.324241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065329ce-350e-4d3e-9aaf-1b0ec97074f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:36.325735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065329ce-350e-4d3e-9aaf-1b0ec97074f7 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:36.355992 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-065329ce-350e-4d3e-9aaf-1b0ec97074f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:36.355992 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:36.355992 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:36.355992 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:36.355992 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:36.355992 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:36.355992 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:36.355992 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:36.355992 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:36.355992 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:36.355992 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:36.355992 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:36.355992 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:36.355992 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:36.355992 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:36.355992 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:36.358302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:36.358302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:36.358302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:36.358302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:36.358302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:36.358302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:36.358302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:36.358302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:36.358302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:36.358302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:36.358302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:36.358302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:36.358302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:36.358302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:36.358302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:36.358302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:36.358302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:36.360397 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:36.360397 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:36.360397 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:36.362674 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 419/821] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:29:36.363507 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065329ce-350e-4d3e-9aaf-1b0ec97074f7 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:36.490442 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b1466e-f73b-42f1-baf1-b102e2c96731 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:36.545459 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065329ce-350e-4d3e-9aaf-1b0ec97074f7 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:36.556323 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:36.566557 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d0aee6-1d50-4763-8df6-74b29b84c9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:36.567185 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0aee6-1d50-4763-8df6-74b29b84c9e8 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:36.567470 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0aee6-1d50-4763-8df6-74b29b84c9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:36.567745 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0aee6-1d50-4763-8df6-74b29b84c9e8 None admin] PATH_INFO: `/v3/services` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:36.568388 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d0aee6-1d50-4763-8df6-74b29b84c9e8 None admin] RBAC: Adding query filter params (type=network) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:36.568693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d0aee6-1d50-4763-8df6-74b29b84c9e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:36.570142 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d0aee6-1d50-4763-8df6-74b29b84c9e8 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:36.583218 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 403/822] 10.222.0.52 () {56 vars in 1112 bytes} [Mon Aug 29 22:29:36 2022] GET /identity/v3/services?type=network => generated 350 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:36.583909 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba183656-2179-4da9-8d8c-e680e66eae63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:36.584623 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba183656-2179-4da9-8d8c-e680e66eae63 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:36.584928 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba183656-2179-4da9-8d8c-e680e66eae63 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:36.585198 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba183656-2179-4da9-8d8c-e680e66eae63 None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:36.585837 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba183656-2179-4da9-8d8c-e680e66eae63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:36.586135 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba183656-2179-4da9-8d8c-e680e66eae63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:36.587492 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba183656-2179-4da9-8d8c-e680e66eae63 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:36.593690 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d0aee6-1d50-4763-8df6-74b29b84c9e8 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:36.600489 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 420/823] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:36.605891 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba183656-2179-4da9-8d8c-e680e66eae63 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:36.720644 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:36.753749 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba183656-2179-4da9-8d8c-e680e66eae63 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:36.790512 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d0aee6-1d50-4763-8df6-74b29b84c9e8 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:36.800939 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7592436e-ec7f-444f-abb7-dc4a4cb2498d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:36.801702 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7592436e-ec7f-444f-abb7-dc4a4cb2498d None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:36.802059 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7592436e-ec7f-444f-abb7-dc4a4cb2498d None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:36.802412 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7592436e-ec7f-444f-abb7-dc4a4cb2498d None admin] PATH_INFO: `/v3/users/nova` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:36.811672 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7592436e-ec7f-444f-abb7-dc4a4cb2498d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:36.813311 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7592436e-ec7f-444f-abb7-dc4a4cb2498d None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:36.827554 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-7592436e-ec7f-444f-abb7-dc4a4cb2498d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 22:29:36.827554 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:36.827554 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:36.827554 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:36.827554 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:36.827554 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:36.827554 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:36.827554 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:36.827554 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:36.827554 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:36.827554 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:36.827554 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:36.827554 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:29:36.827554 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:29:36.827554 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:29:36.827554 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:29:36.828555 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:36.828555 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:36.828555 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:29:36.828555 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:36.828555 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:29:36.828555 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:36.828555 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:36.828555 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:36.828555 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:36.828555 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:36.828555 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:36.828555 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:36.828555 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:36.828555 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:36.828555 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:36.828555 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:36.828555 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:36.828555 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:36.829531 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:36.829531 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:36.829531 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:29:36.829531 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:29:36.829531 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:29:36.829531 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:29:36.829531 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:29:36.829531 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:29:36.829531 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 22:29:36.829531 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:36.829531 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 421/824] 10.222.0.52 () {56 vars in 1093 bytes} [Mon Aug 29 22:29:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:36.834441 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7592436e-ec7f-444f-abb7-dc4a4cb2498d None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:36.880771 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b26945-d12c-474b-95d7-070f0b1fbefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:36.881396 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b26945-d12c-474b-95d7-070f0b1fbefd None admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:36.881583 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:36.881686 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b26945-d12c-474b-95d7-070f0b1fbefd None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:36.881941 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b26945-d12c-474b-95d7-070f0b1fbefd None admin] PATH_INFO: `/v3/endpoints` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:36.882648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b26945-d12c-474b-95d7-070f0b1fbefd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:36.884000 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b26945-d12c-474b-95d7-070f0b1fbefd None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:36.979369 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 404/825] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:29:36 2022] POST /identity/v3/endpoints => generated 337 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:29:36.980002 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7592436e-ec7f-444f-abb7-dc4a4cb2498d None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:37.027492 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a80824c-db4c-4000-acf0-29c86e72053c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:37.028524 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a80824c-db4c-4000-acf0-29c86e72053c None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:37.029062 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a80824c-db4c-4000-acf0-29c86e72053c None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:37.029539 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a80824c-db4c-4000-acf0-29c86e72053c None admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:37.030499 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a80824c-db4c-4000-acf0-29c86e72053c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:37.031061 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a80824c-db4c-4000-acf0-29c86e72053c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:37.032788 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a80824c-db4c-4000-acf0-29c86e72053c None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:37.062054 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 422/826] 10.222.0.52 () {56 vars in 1133 bytes} [Mon Aug 29 22:29:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:37.068074 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b26945-d12c-474b-95d7-070f0b1fbefd None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:37.217331 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b26945-d12c-474b-95d7-070f0b1fbefd None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:37.245037 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:37.260175 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de9329-12ed-444e-a494-01674343d270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:37.260829 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de9329-12ed-444e-a494-01674343d270 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:37.261111 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de9329-12ed-444e-a494-01674343d270 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:37.261379 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de9329-12ed-444e-a494-01674343d270 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:37.336567 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75de9329-12ed-444e-a494-01674343d270 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:37.337974 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75de9329-12ed-444e-a494-01674343d270 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:37.449054 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 405/827] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:29:37 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 22:29:37.450586 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a80824c-db4c-4000-acf0-29c86e72053c None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:37.557959 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:37.570688 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a80824c-db4c-4000-acf0-29c86e72053c None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:37.589623 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1ecc4-f524-440c-baa5-6590fd617c45 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:37.589885 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1ecc4-f524-440c-baa5-6590fd617c45 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:37.590107 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1ecc4-f524-440c-baa5-6590fd617c45 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:37.591694 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 406/828] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:29:37.597605 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61084963-ea38-4779-b5e5-453a86a0ae39 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:37.597884 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61084963-ea38-4779-b5e5-453a86a0ae39 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:37.598169 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61084963-ea38-4779-b5e5-453a86a0ae39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:37.609464 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5153b359-b322-419a-8093-83c9ab8e9cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:37.610465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5153b359-b322-419a-8093-83c9ab8e9cb6 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:37.610998 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5153b359-b322-419a-8093-83c9ab8e9cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:37.612032 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5153b359-b322-419a-8093-83c9ab8e9cb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:37.618088 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-61084963-ea38-4779-b5e5-453a86a0ae39 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:37.619330 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-61084963-ea38-4779-b5e5-453a86a0ae39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:37.623929 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5153b359-b322-419a-8093-83c9ab8e9cb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:37.625649 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5153b359-b322-419a-8093-83c9ab8e9cb6 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:37.639054 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:37.644064 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-5153b359-b322-419a-8093-83c9ab8e9cb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:37.644064 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:37.644064 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:37.644064 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:37.644064 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:37.644064 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:37.644064 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:37.644064 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:37.644064 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:37.644064 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:37.644064 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:37.644064 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:37.644064 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:37.644064 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:37.644064 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:37.644064 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:37.645033 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:37.645033 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:37.645033 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:37.645033 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:37.645033 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:37.645033 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:37.645033 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:37.645033 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:37.645033 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:37.645033 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:37.645033 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:37.645033 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:37.645033 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:37.645033 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:37.645033 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:37.645033 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:37.645033 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:37.645933 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:37.645933 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:37.645933 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:37.647994 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 423/829] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:37.654938 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5153b359-b322-419a-8093-83c9ab8e9cb6 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:37.700754 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 407/830] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:37 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:29:37.778896 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ff455-41ef-48fc-a0c4-147e2add3994 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:37.779493 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ff455-41ef-48fc-a0c4-147e2add3994 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:37.779987 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ff455-41ef-48fc-a0c4-147e2add3994 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:37.781625 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5153b359-b322-419a-8093-83c9ab8e9cb6 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:37.782059 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 408/831] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:29:37.788198 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f6506-f3c4-4488-8481-76d19a6911bd None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:37.788780 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f6506-f3c4-4488-8481-76d19a6911bd None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:37.789261 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f6506-f3c4-4488-8481-76d19a6911bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:37.809351 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-775f6506-f3c4-4488-8481-76d19a6911bd None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:37.811482 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-775f6506-f3c4-4488-8481-76d19a6911bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:37.819590 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b9e982-bb8a-43b9-9a06-98097ab797c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:37.820197 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9e982-bb8a-43b9-9a06-98097ab797c7 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:37.820479 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9e982-bb8a-43b9-9a06-98097ab797c7 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:37.820772 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9e982-bb8a-43b9-9a06-98097ab797c7 None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:37.821418 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b9e982-bb8a-43b9-9a06-98097ab797c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:37.821721 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b9e982-bb8a-43b9-9a06-98097ab797c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:37.823083 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b9e982-bb8a-43b9-9a06-98097ab797c7 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:37.824707 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:37.836751 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 424/832] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:37.843885 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b9e982-bb8a-43b9-9a06-98097ab797c7 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:37.904893 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 409/833] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:37 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:29:37.969142 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b9e982-bb8a-43b9-9a06-98097ab797c7 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:37.993269 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f6506-f3c4-4488-8481-76d19a6911bd None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:38.008086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3a987d-7bbd-4124-a70b-083f46100547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:38.008746 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a987d-7bbd-4124-a70b-083f46100547 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:38.009086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a987d-7bbd-4124-a70b-083f46100547 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:38.009363 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a987d-7bbd-4124-a70b-083f46100547 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:38.016560 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3a987d-7bbd-4124-a70b-083f46100547 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:38.019325 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3a987d-7bbd-4124-a70b-083f46100547 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:38.031258 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-2c3a987d-7bbd-4124-a70b-083f46100547 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:38.031258 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:38.031258 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:38.031258 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:38.031258 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:38.031258 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:38.031258 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:38.031258 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:38.031258 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:38.031258 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:38.031258 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:38.031258 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:38.031258 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:29:38.031258 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:29:38.031258 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:29:38.032182 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:29:38.032182 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:38.032182 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:38.032182 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:38.032182 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:38.032182 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:38.032182 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:38.032182 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:38.032182 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:38.032182 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:38.032182 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:38.032182 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:38.032182 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:38.032182 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:38.032182 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:38.032182 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:38.032182 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:38.033267 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:29:38.033267 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:29:38.033267 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:29:38.033267 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:29:38.033267 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:29:38.033267 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:29:38.033267 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:38.033267 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:38.033267 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 425/834] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:29:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:38.038682 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3a987d-7bbd-4124-a70b-083f46100547 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:38.108499 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:38.112987 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:38.131135 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f6506-f3c4-4488-8481-76d19a6911bd None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:38.162948 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3a987d-7bbd-4124-a70b-083f46100547 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:38.167496 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355aaf76-43b1-4fd4-8491-06a45bd840cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:38.168140 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355aaf76-43b1-4fd4-8491-06a45bd840cd None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:38.168421 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355aaf76-43b1-4fd4-8491-06a45bd840cd None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:38.168695 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355aaf76-43b1-4fd4-8491-06a45bd840cd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:38.181104 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355aaf76-43b1-4fd4-8491-06a45bd840cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:29:38.186083 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-355aaf76-43b1-4fd4-8491-06a45bd840cd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:29:38.186083 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:38.186083 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:38.186083 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:38.186083 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:38.186083 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:38.186083 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:38.186083 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:38.186083 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:38.186083 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:38.186083 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:38.186083 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:38.186083 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:29:38.186083 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:29:38.186083 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:29:38.186083 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:29:38.187203 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:29:38.187203 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:29:38.187203 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:29:38.187203 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:29:38.187203 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:38.187203 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:38.187203 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:38.187203 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:38.187203 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:38.187203 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:38.187203 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:38.187203 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:38.187203 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:38.187203 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:38.187203 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:38.187203 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:38.188619 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:38.188619 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:38.188619 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:38.188619 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:38.188619 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:29:38.188619 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:29:38.188619 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:29:38.188619 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:29:38.188619 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:29:38.188619 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:29:38.188619 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:29:38.188619 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:38.188619 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 410/835] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Aug 29 22:29:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:29:38.188619 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7bf4c-d933-4ae9-8ec5-945e8e2cabaf None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:38.190160 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7bf4c-d933-4ae9-8ec5-945e8e2cabaf None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:38.190160 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7bf4c-d933-4ae9-8ec5-945e8e2cabaf None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:38.192119 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 411/836] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:29:38.196936 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb7bf4c-d933-4ae9-8ec5-945e8e2cabaf None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:38.207489 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c3ccf2-d978-4c7b-a97d-b565b5b6e0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:38.208091 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c3ccf2-d978-4c7b-a97d-b565b5b6e0fb None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:38.208373 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c3ccf2-d978-4c7b-a97d-b565b5b6e0fb None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:38.208644 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c3ccf2-d978-4c7b-a97d-b565b5b6e0fb None admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:38.210900 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c3ccf2-d978-4c7b-a97d-b565b5b6e0fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:38.211104 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c3ccf2-d978-4c7b-a97d-b565b5b6e0fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:38.212543 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c3ccf2-d978-4c7b-a97d-b565b5b6e0fb None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:38.224932 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 426/837] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Aug 29 22:29:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:38.225818 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c3ccf2-d978-4c7b-a97d-b565b5b6e0fb None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:38.301884 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb7bf4c-d933-4ae9-8ec5-945e8e2cabaf None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:38.326000 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c3ccf2-d978-4c7b-a97d-b565b5b6e0fb None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:38.336891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708df2cf-a2f9-48c1-a058-06b9a1d34e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:38.337489 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708df2cf-a2f9-48c1-a058-06b9a1d34e7c None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:38.337767 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708df2cf-a2f9-48c1-a058-06b9a1d34e7c None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:38.338035 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708df2cf-a2f9-48c1-a058-06b9a1d34e7c None admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:38.338690 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708df2cf-a2f9-48c1-a058-06b9a1d34e7c None admin] RBAC: Adding query filter params (name=admin) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:38.338991 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708df2cf-a2f9-48c1-a058-06b9a1d34e7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:38.340362 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708df2cf-a2f9-48c1-a058-06b9a1d34e7c None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:38.340806 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:38.348926 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 412/838] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:29:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:29:38.354036 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907e8d0-924a-44a9-898e-01a4abac64e5 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:38.354320 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907e8d0-924a-44a9-898e-01a4abac64e5 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:38.354602 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907e8d0-924a-44a9-898e-01a4abac64e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:38.361082 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2591acf6-dbf9-46db-9cf5-22927101be30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:38.362789 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591acf6-dbf9-46db-9cf5-22927101be30 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:38.363072 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591acf6-dbf9-46db-9cf5-22927101be30 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:38.363345 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591acf6-dbf9-46db-9cf5-22927101be30 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:38.374825 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2591acf6-dbf9-46db-9cf5-22927101be30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:29:38.380450 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-2591acf6-dbf9-46db-9cf5-22927101be30 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:29:38.380450 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:38.380450 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:38.380450 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:38.380450 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:38.380450 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:38.380450 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:38.380450 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:38.380450 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:38.380450 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:38.380450 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:38.380450 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:38.380450 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:29:38.380450 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:29:38.380450 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:29:38.380450 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:29:38.381532 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:29:38.381532 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:29:38.381532 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:29:38.381532 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:29:38.381532 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:38.381532 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:38.381532 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:38.381532 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:38.381532 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:38.381532 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:38.381532 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:38.381532 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:38.381532 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:38.381532 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:38.381532 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:38.381532 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:38.382676 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:38.382676 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:38.382676 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:38.382676 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:38.382676 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:29:38.382676 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:29:38.382676 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:29:38.382676 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:29:38.382676 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:29:38.382676 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:29:38.382676 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:29:38.382676 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:38.382676 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 427/839] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Aug 29 22:29:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:38.383802 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2591acf6-dbf9-46db-9cf5-22927101be30 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:38.482761 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:38.489408 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-1907e8d0-924a-44a9-898e-01a4abac64e5 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:38.492809 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-1907e8d0-924a-44a9-898e-01a4abac64e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:38.494501 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2591acf6-dbf9-46db-9cf5-22927101be30 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:38.529305 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5371026c-5bff-4cf2-88fe-df4dbc04c241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:38.529907 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371026c-5bff-4cf2-88fe-df4dbc04c241 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:38.530189 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371026c-5bff-4cf2-88fe-df4dbc04c241 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:38.530466 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371026c-5bff-4cf2-88fe-df4dbc04c241 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:38.531187 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5371026c-5bff-4cf2-88fe-df4dbc04c241 None admin] RBAC: Adding query filter params (role.id=882f35bd847a406eb2222aeafc517a46, scope.project.id=ee84c3340f2641ad963655a7a212ef96, user.id=bdbf961e02934e88af3a72d6218f32fa) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:38.531501 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5371026c-5bff-4cf2-88fe-df4dbc04c241 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:38.532989 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5371026c-5bff-4cf2-88fe-df4dbc04c241 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:38.548885 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 428/840] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:29:38 2022] GET /identity/v3/role_assignments?user.id=bdbf961e02934e88af3a72d6218f32fa&scope.project.id=ee84c3340f2641ad963655a7a212ef96&role.id=882f35bd847a406eb2222aeafc517a46 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:38.548885 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5371026c-5bff-4cf2-88fe-df4dbc04c241 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:38.580795 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 413/841] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:38 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:29:38.580795 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1907e8d0-924a-44a9-898e-01a4abac64e5 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:38.616196 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:38.710337 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5371026c-5bff-4cf2-88fe-df4dbc04c241 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:38.710662 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1907e8d0-924a-44a9-898e-01a4abac64e5 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:38.750101 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e04672-74a2-4552-9278-f1122fde9eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:38.750307 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e04672-74a2-4552-9278-f1122fde9eca None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:38.750516 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e04672-74a2-4552-9278-f1122fde9eca None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:38.750774 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e04672-74a2-4552-9278-f1122fde9eca None admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:38.751427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e04672-74a2-4552-9278-f1122fde9eca None admin] RBAC: Adding query filter params (name=service) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:38.751729 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e04672-74a2-4552-9278-f1122fde9eca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:38.753661 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e04672-74a2-4552-9278-f1122fde9eca None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:38.756013 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:38.758399 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9407c1d9-911d-4b48-b715-76321344126d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:38.759290 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9407c1d9-911d-4b48-b715-76321344126d None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:38.759763 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9407c1d9-911d-4b48-b715-76321344126d None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:38.760241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9407c1d9-911d-4b48-b715-76321344126d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:38.763865 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 414/842] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Aug 29 22:29:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:38.771152 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e04672-74a2-4552-9278-f1122fde9eca None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:38.774216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9407c1d9-911d-4b48-b715-76321344126d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:38.774888 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9407c1d9-911d-4b48-b715-76321344126d None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:38.791843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-9407c1d9-911d-4b48-b715-76321344126d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:38.791843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:38.791843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:38.791843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:38.791843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:38.791843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:38.791843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:38.791843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:38.791843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:38.791843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:38.791843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:38.791843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:38.791843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:38.791843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:38.791843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:38.791843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:38.792947 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:38.792947 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:38.792947 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:38.792947 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:38.792947 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:38.792947 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:38.792947 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:38.792947 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:38.792947 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:38.792947 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:38.792947 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:38.792947 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:38.792947 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:38.792947 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:38.792947 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:38.792947 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:38.792947 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:38.793822 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:38.793822 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:38.793822 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:38.793822 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 429/843] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:38.798933 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9407c1d9-911d-4b48-b715-76321344126d None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:38.893994 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e04672-74a2-4552-9278-f1122fde9eca None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:38.921065 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:38.934288 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7d44be-1bf7-4f94-b5a7-1b2b448281d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:38.934949 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d44be-1bf7-4f94-b5a7-1b2b448281d1 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:38.935294 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d44be-1bf7-4f94-b5a7-1b2b448281d1 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:38.935927 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d44be-1bf7-4f94-b5a7-1b2b448281d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:38.936097 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9407c1d9-911d-4b48-b715-76321344126d None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:38.948186 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7d44be-1bf7-4f94-b5a7-1b2b448281d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:38.949671 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7d44be-1bf7-4f94-b5a7-1b2b448281d1 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:38.965448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-ce7d44be-1bf7-4f94-b5a7-1b2b448281d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:38.965448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:38.965448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:38.965448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:38.965448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:38.965448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:38.965448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:38.965448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:38.965448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:38.965448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:38.965448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:38.965448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:38.965448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:38.965448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:38.965448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:38.965448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:38.966648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:38.966648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:38.966648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:38.966648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:38.966648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:38.966648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:38.966648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:38.966648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:38.966648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:38.966648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:38.966648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:38.966648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:38.966648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:38.966648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:38.966648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:38.966648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:38.966648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:38.967987 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:38.967987 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:38.967987 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:38.972425 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 415/844] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:29:38.976066 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7d44be-1bf7-4f94-b5a7-1b2b448281d1 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:38.976174 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ee1af8-9bda-48f6-82f9-46aa7076f80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:38.976351 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee1af8-9bda-48f6-82f9-46aa7076f80c None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:38.976635 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee1af8-9bda-48f6-82f9-46aa7076f80c None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:38.976930 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee1af8-9bda-48f6-82f9-46aa7076f80c None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:38.977574 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ee1af8-9bda-48f6-82f9-46aa7076f80c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:38.977874 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ee1af8-9bda-48f6-82f9-46aa7076f80c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:38.979231 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ee1af8-9bda-48f6-82f9-46aa7076f80c None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:38.992160 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 430/845] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:38.993029 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ee1af8-9bda-48f6-82f9-46aa7076f80c None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:39.039543 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:39.094219 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7d44be-1bf7-4f94-b5a7-1b2b448281d1 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:39.103842 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ee1af8-9bda-48f6-82f9-46aa7076f80c None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:39.130806 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6fce99-e327-4e1d-8dd5-38af0642b273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:39.131446 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6fce99-e327-4e1d-8dd5-38af0642b273 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:39.131725 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6fce99-e327-4e1d-8dd5-38af0642b273 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:39.132000 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6fce99-e327-4e1d-8dd5-38af0642b273 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:39.132581 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6fce99-e327-4e1d-8dd5-38af0642b273 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:39.133992 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6fce99-e327-4e1d-8dd5-38af0642b273 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:39.142324 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89435bc6-f153-487d-8a23-12c8815952cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:39.143598 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89435bc6-f153-487d-8a23-12c8815952cf None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:39.143878 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-ae6fce99-e327-4e1d-8dd5-38af0642b273 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 22:29:39.143878 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:39.143878 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:29:39.143878 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:29:39.143878 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:29:39.143878 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:29:39.143878 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:29:39.143878 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:29:39.143878 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 22:29:39.143878 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:39.143878 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:29:39.143878 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:39.143878 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:39.143878 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:39.143878 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:39.143878 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:39.143878 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:39.145220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:39.145220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:39.145220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:39.145220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:39.145220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:39.145220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:39.145220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:29:39.145220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:29:39.145220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:29:39.145220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:29:39.145220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:39.145220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:39.145220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:39.145220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:39.145220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:39.145220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:39.146049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:39.146049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:39.146049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:39.146049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:39.146049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:39.146049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:39.146049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:39.146049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:39.146049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:39.146049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:39.146049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:29:39.146049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:29:39.146049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 22:29:39.146049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:39.146782 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89435bc6-f153-487d-8a23-12c8815952cf None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:39.146782 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89435bc6-f153-487d-8a23-12c8815952cf None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:39.146782 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89435bc6-f153-487d-8a23-12c8815952cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:39.146782 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89435bc6-f153-487d-8a23-12c8815952cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:39.147020 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 416/846] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Aug 29 22:29:38 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:39.147020 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6fce99-e327-4e1d-8dd5-38af0642b273 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:39.147150 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89435bc6-f153-487d-8a23-12c8815952cf None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:39.160222 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 431/847] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:39.161030 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89435bc6-f153-487d-8a23-12c8815952cf None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:39.177062 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:39.278812 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6fce99-e327-4e1d-8dd5-38af0642b273 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:39.280389 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89435bc6-f153-487d-8a23-12c8815952cf None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:39.295805 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:39.318331 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41035e13-d20b-4a1c-a040-3398728fddcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:39.318957 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41035e13-d20b-4a1c-a040-3398728fddcd None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:39.319238 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41035e13-d20b-4a1c-a040-3398728fddcd None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:39.319512 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41035e13-d20b-4a1c-a040-3398728fddcd None admin] PATH_INFO: `/v3/users/swift` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:39.325696 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e5e72-a5b7-4d9b-99f2-ca1b2f8c7172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:39.326295 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e5e72-a5b7-4d9b-99f2-ca1b2f8c7172 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:39.326574 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e5e72-a5b7-4d9b-99f2-ca1b2f8c7172 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:39.327320 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41035e13-d20b-4a1c-a040-3398728fddcd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:39.328808 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41035e13-d20b-4a1c-a040-3398728fddcd None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:39.332575 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e5e72-a5b7-4d9b-99f2-ca1b2f8c7172 None admin] PATH_INFO: `/v3/services` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:39.333261 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8e5e72-a5b7-4d9b-99f2-ca1b2f8c7172 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:39.333574 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8e5e72-a5b7-4d9b-99f2-ca1b2f8c7172 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:39.334936 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8e5e72-a5b7-4d9b-99f2-ca1b2f8c7172 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:39.344297 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 432/848] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Aug 29 22:29:39 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:39.344297 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e5e72-a5b7-4d9b-99f2-ca1b2f8c7172 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:39.344544 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-41035e13-d20b-4a1c-a040-3398728fddcd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 22:29:39.344544 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:39.344544 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:39.344544 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:39.344544 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:39.344544 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:39.344544 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:39.344544 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:39.344544 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:39.344544 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:39.344544 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:39.344544 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:39.344544 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:29:39.344544 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:29:39.344544 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:29:39.344544 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:29:39.345449 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:39.345449 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:39.345449 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:29:39.345449 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:39.345449 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:29:39.345449 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:39.345449 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:39.345449 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:39.345449 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:39.345449 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:39.345449 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:39.345449 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:39.345449 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:39.345449 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:39.345449 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:39.345449 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:39.345449 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:39.345449 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:39.346561 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:39.346561 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:39.346561 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:29:39.346561 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:29:39.346561 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:29:39.346561 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:29:39.346561 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:29:39.346561 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:29:39.346561 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 22:29:39.346561 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:39.346561 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 417/849] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Aug 29 22:29:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:39.349471 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41035e13-d20b-4a1c-a040-3398728fddcd None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:39.423563 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:39.449622 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e5e72-a5b7-4d9b-99f2-ca1b2f8c7172 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:39.456560 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41035e13-d20b-4a1c-a040-3398728fddcd None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:39.490229 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56413bbb-54e9-4bfa-84ba-f110396a69f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:39.490862 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56413bbb-54e9-4bfa-84ba-f110396a69f8 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:39.491137 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56413bbb-54e9-4bfa-84ba-f110396a69f8 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:39.491402 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56413bbb-54e9-4bfa-84ba-f110396a69f8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:39.492230 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d6bac-53c6-4d16-82ac-f629fb0a3462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:39.492865 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d6bac-53c6-4d16-82ac-f629fb0a3462 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:39.493158 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d6bac-53c6-4d16-82ac-f629fb0a3462 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:39.493403 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d6bac-53c6-4d16-82ac-f629fb0a3462 None admin] PATH_INFO: `/v3/services` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:39.494031 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70d6bac-53c6-4d16-82ac-f629fb0a3462 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:39.494327 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70d6bac-53c6-4d16-82ac-f629fb0a3462 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:39.495672 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70d6bac-53c6-4d16-82ac-f629fb0a3462 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:39.500245 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 418/850] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Aug 29 22:29:39 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:39.501056 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d6bac-53c6-4d16-82ac-f629fb0a3462 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:39.508975 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56413bbb-54e9-4bfa-84ba-f110396a69f8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:39.511346 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56413bbb-54e9-4bfa-84ba-f110396a69f8 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:39.530132 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-56413bbb-54e9-4bfa-84ba-f110396a69f8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 22:29:39.530132 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:39.530132 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:39.530132 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:39.530132 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:39.530132 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:39.530132 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:39.530132 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:39.530132 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:39.530132 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:39.530132 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:39.530132 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:39.530132 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:29:39.530132 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:29:39.530132 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:29:39.530132 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:29:39.531410 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:39.531410 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:39.531410 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:29:39.531410 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:39.531410 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:29:39.531410 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:39.531410 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:39.531410 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:39.531410 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:39.531410 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:39.531410 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:39.531410 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:39.531410 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:39.531410 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:39.531410 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:39.531410 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:39.531410 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:39.531410 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:39.532514 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:39.532514 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:39.532514 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:29:39.532514 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:29:39.532514 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:29:39.532514 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:29:39.532514 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:29:39.532514 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:29:39.532514 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 22:29:39.532514 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:39.547025 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 433/851] 10.222.0.52 () {56 vars in 1117 bytes} [Mon Aug 29 22:29:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:39.547964 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56413bbb-54e9-4bfa-84ba-f110396a69f8 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:39.560810 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:39.643605 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d6bac-53c6-4d16-82ac-f629fb0a3462 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:39.683153 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6c4182-0ab7-4c51-8dd7-6da999da172b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:39.683608 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c4182-0ab7-4c51-8dd7-6da999da172b None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:39.683883 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c4182-0ab7-4c51-8dd7-6da999da172b None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:39.684150 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c4182-0ab7-4c51-8dd7-6da999da172b None admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:39.684908 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6c4182-0ab7-4c51-8dd7-6da999da172b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:39.685381 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6c4182-0ab7-4c51-8dd7-6da999da172b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:39.686944 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6c4182-0ab7-4c51-8dd7-6da999da172b None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:39.706784 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 419/852] 10.222.0.52 () {56 vars in 1135 bytes} [Mon Aug 29 22:29:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:39.715253 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6c4182-0ab7-4c51-8dd7-6da999da172b None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:39.760367 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56413bbb-54e9-4bfa-84ba-f110396a69f8 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:39.818755 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:39.825474 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6c4182-0ab7-4c51-8dd7-6da999da172b None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:39.826406 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e68b9b-e6d4-4f77-8743-256fde3894b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:39.827642 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e68b9b-e6d4-4f77-8743-256fde3894b4 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:39.828323 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e68b9b-e6d4-4f77-8743-256fde3894b4 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:39.829111 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e68b9b-e6d4-4f77-8743-256fde3894b4 None admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:39.831138 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e68b9b-e6d4-4f77-8743-256fde3894b4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:39.831887 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e68b9b-e6d4-4f77-8743-256fde3894b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:39.834527 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e68b9b-e6d4-4f77-8743-256fde3894b4 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:39.837675 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:39.866919 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de60d1-b22b-4c52-90d2-a081277f36ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:39.867662 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de60d1-b22b-4c52-90d2-a081277f36ca None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:39.867996 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de60d1-b22b-4c52-90d2-a081277f36ca None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:39.868327 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de60d1-b22b-4c52-90d2-a081277f36ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:39.870906 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 434/853] 10.222.0.52 () {56 vars in 1149 bytes} [Mon Aug 29 22:29:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:39.876849 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e68b9b-e6d4-4f77-8743-256fde3894b4 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:39.957541 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5de60d1-b22b-4c52-90d2-a081277f36ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:39.958983 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5de60d1-b22b-4c52-90d2-a081277f36ca None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:40.020956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e68b9b-e6d4-4f77-8743-256fde3894b4 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:40.067306 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3e6f40-743b-4017-98bf-395894870df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:40.068302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e6f40-743b-4017-98bf-395894870df1 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:40.068837 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e6f40-743b-4017-98bf-395894870df1 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:40.069308 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e6f40-743b-4017-98bf-395894870df1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:40.089246 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:40.101591 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 420/854] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:29:39 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 22:29:40.107231 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de60d1-b22b-4c52-90d2-a081277f36ca None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:40.140878 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:40.156102 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3e6f40-743b-4017-98bf-395894870df1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:40.158691 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3e6f40-743b-4017-98bf-395894870df1 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:40.238853 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de60d1-b22b-4c52-90d2-a081277f36ca None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:40.286343 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0117c8-de15-4213-aa0d-98bd2eae03ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:40.288043 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0117c8-de15-4213-aa0d-98bd2eae03ae None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:40.288533 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0117c8-de15-4213-aa0d-98bd2eae03ae None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:40.289404 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0117c8-de15-4213-aa0d-98bd2eae03ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:40.301723 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 435/855] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:29:39 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 22:29:40.302667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3e6f40-743b-4017-98bf-395894870df1 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:40.304054 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0117c8-de15-4213-aa0d-98bd2eae03ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:40.306990 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0117c8-de15-4213-aa0d-98bd2eae03ae None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:40.327651 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-cc0117c8-de15-4213-aa0d-98bd2eae03ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:40.327651 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:40.327651 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:40.327651 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:40.327651 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:40.327651 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:40.327651 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:40.327651 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:40.327651 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:40.327651 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:40.327651 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:40.327651 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:40.327651 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:40.327651 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:40.327651 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:40.327651 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:40.328641 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:40.328641 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:40.328641 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:40.328641 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:40.328641 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:40.328641 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:40.328641 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:40.328641 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:40.328641 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:40.328641 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:40.328641 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:40.328641 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:40.328641 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:40.328641 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:40.328641 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:40.328641 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:40.328641 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:40.329588 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:40.329588 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:40.329588 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:40.332862 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 421/856] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:29:40.336380 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0117c8-de15-4213-aa0d-98bd2eae03ae None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:40.399079 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:40.408196 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:40.417813 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3e6f40-743b-4017-98bf-395894870df1 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:40.455593 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c110072f-d1db-46c0-a31d-730919c347d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:40.456254 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110072f-d1db-46c0-a31d-730919c347d6 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:40.456530 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110072f-d1db-46c0-a31d-730919c347d6 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:40.456845 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110072f-d1db-46c0-a31d-730919c347d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:40.457252 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0117c8-de15-4213-aa0d-98bd2eae03ae None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:40.467904 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c110072f-d1db-46c0-a31d-730919c347d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:40.469492 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c110072f-d1db-46c0-a31d-730919c347d6 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:40.490174 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-c110072f-d1db-46c0-a31d-730919c347d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:40.490174 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:40.490174 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:40.490174 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:40.490174 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:40.490174 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:40.490174 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:40.490174 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:40.490174 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:40.490174 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:40.490174 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:40.490174 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:40.490174 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:40.490174 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:40.490174 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:40.490174 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:40.491155 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:40.491155 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:40.491155 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:40.491155 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:40.491155 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:40.491155 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:40.491155 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:40.491155 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:40.491155 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:40.491155 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:40.491155 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:40.491155 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:40.491155 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:40.491155 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:40.491155 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:40.491155 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:40.491155 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:40.492076 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:40.492076 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:40.492076 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:40.496351 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 436/857] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:40.497336 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c110072f-d1db-46c0-a31d-730919c347d6 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:40.499273 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28cac2b-aac8-4a96-a639-7f54ea865a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:40.500313 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28cac2b-aac8-4a96-a639-7f54ea865a51 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:40.500828 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28cac2b-aac8-4a96-a639-7f54ea865a51 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:40.502816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28cac2b-aac8-4a96-a639-7f54ea865a51 None admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:40.504090 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28cac2b-aac8-4a96-a639-7f54ea865a51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:40.504586 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28cac2b-aac8-4a96-a639-7f54ea865a51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:40.506167 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28cac2b-aac8-4a96-a639-7f54ea865a51 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:40.523717 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 422/858] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:29:40.530049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28cac2b-aac8-4a96-a639-7f54ea865a51 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:40.635356 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c110072f-d1db-46c0-a31d-730919c347d6 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:40.645899 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28cac2b-aac8-4a96-a639-7f54ea865a51 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:40.655732 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:40.674685 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178714d3-d62e-44e3-bd53-bf19b13df542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:40.675827 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178714d3-d62e-44e3-bd53-bf19b13df542 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:40.676276 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178714d3-d62e-44e3-bd53-bf19b13df542 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:40.676687 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178714d3-d62e-44e3-bd53-bf19b13df542 None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:40.677491 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178714d3-d62e-44e3-bd53-bf19b13df542 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:40.677948 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178714d3-d62e-44e3-bd53-bf19b13df542 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:40.679702 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178714d3-d62e-44e3-bd53-bf19b13df542 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:40.682016 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd490c-cf11-46bf-a56c-6c3f3f437171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:40.682645 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd490c-cf11-46bf-a56c-6c3f3f437171 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:40.683283 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd490c-cf11-46bf-a56c-6c3f3f437171 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:40.683283 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd490c-cf11-46bf-a56c-6c3f3f437171 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:40.691035 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cd490c-cf11-46bf-a56c-6c3f3f437171 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:40.692535 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cd490c-cf11-46bf-a56c-6c3f3f437171 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:40.701246 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 437/859] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:40.702320 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178714d3-d62e-44e3-bd53-bf19b13df542 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:40.705299 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-74cd490c-cf11-46bf-a56c-6c3f3f437171 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:40.705299 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:40.705299 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:40.705299 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:40.705299 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:40.705299 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:40.705299 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:40.705299 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:40.705299 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:40.705299 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:40.705299 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:40.705299 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:40.705299 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:29:40.705299 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:29:40.705299 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:29:40.706531 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:29:40.706531 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:40.706531 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:40.706531 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:40.706531 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:40.706531 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:40.706531 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:40.706531 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:40.706531 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:40.706531 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:40.706531 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:40.706531 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:40.706531 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:40.706531 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:40.706531 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:40.706531 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:40.706531 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:40.707693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:29:40.707693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:29:40.707693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:29:40.707693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:29:40.707693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:29:40.707693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:29:40.707693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:40.707693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:40.709456 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:40.713106 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 423/860] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:29:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:40.713972 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd490c-cf11-46bf-a56c-6c3f3f437171 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:40.816450 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178714d3-d62e-44e3-bd53-bf19b13df542 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:40.833785 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd490c-cf11-46bf-a56c-6c3f3f437171 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:40.854767 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7db421-fc24-4bbb-bba0-e278186c6ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:40.855399 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7db421-fc24-4bbb-bba0-e278186c6ecb None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:40.855682 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7db421-fc24-4bbb-bba0-e278186c6ecb None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:40.856055 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7db421-fc24-4bbb-bba0-e278186c6ecb None admin] PATH_INFO: `/v3/projects/service` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:40.864005 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7db421-fc24-4bbb-bba0-e278186c6ecb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:40.865752 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7db421-fc24-4bbb-bba0-e278186c6ecb None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:40.870075 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64e8f6c-3a4c-4578-b882-6801a3d8ef46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:40.870684 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e8f6c-3a4c-4578-b882-6801a3d8ef46 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:40.870962 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e8f6c-3a4c-4578-b882-6801a3d8ef46 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:40.871227 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e8f6c-3a4c-4578-b882-6801a3d8ef46 None admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:40.871921 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64e8f6c-3a4c-4578-b882-6801a3d8ef46 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:40.872227 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64e8f6c-3a4c-4578-b882-6801a3d8ef46 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:40.873675 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64e8f6c-3a4c-4578-b882-6801a3d8ef46 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:40.878500 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-db7db421-fc24-4bbb-bba0-e278186c6ecb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:40.878500 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:40.878500 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:40.878500 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:40.878500 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:40.878500 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:40.878500 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:40.878500 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:40.878500 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:40.878500 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:40.878500 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:40.878500 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:40.878500 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:29:40.878500 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:29:40.878500 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:29:40.879561 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:29:40.879561 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:40.879561 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:40.879561 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:40.879561 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:40.879561 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:40.879561 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:40.879561 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:40.879561 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:40.879561 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:40.879561 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:40.879561 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:40.879561 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:40.879561 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:40.879561 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:40.879561 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:40.879561 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:40.880591 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:29:40.880591 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:29:40.880591 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:29:40.880591 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:29:40.880591 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:29:40.880591 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:29:40.880591 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:40.880591 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:40.882394 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 438/861] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:29:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:40.883684 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6db0d-4c9d-4484-9153-ed8966089c37 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:40.884112 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6db0d-4c9d-4484-9153-ed8966089c37 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:40.884527 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6db0d-4c9d-4484-9153-ed8966089c37 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:40.886253 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 439/862] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:29:40.887849 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c6db0d-4c9d-4484-9153-ed8966089c37 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:40.892072 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 424/863] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Aug 29 22:29:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:40.893729 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625c6dc-724c-4776-b2aa-a623f1da9beb None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:40.894017 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625c6dc-724c-4776-b2aa-a623f1da9beb None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:40.894291 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625c6dc-724c-4776-b2aa-a623f1da9beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:40.914071 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-0625c6dc-724c-4776-b2aa-a623f1da9beb None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:40.915323 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-0625c6dc-724c-4776-b2aa-a623f1da9beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:40.941696 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:40.962190 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:40.993245 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 425/864] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:40 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:29:40.994172 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0625c6dc-724c-4776-b2aa-a623f1da9beb None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:41.007835 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c6db0d-4c9d-4484-9153-ed8966089c37 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:41.045527 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca59df-e155-4a84-a73f-73bc524f7f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:41.046536 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca59df-e155-4a84-a73f-73bc524f7f10 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:41.047016 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca59df-e155-4a84-a73f-73bc524f7f10 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:41.047511 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca59df-e155-4a84-a73f-73bc524f7f10 None admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:41.048426 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dca59df-e155-4a84-a73f-73bc524f7f10 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:41.049624 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dca59df-e155-4a84-a73f-73bc524f7f10 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:41.051796 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dca59df-e155-4a84-a73f-73bc524f7f10 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:41.069193 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 440/865] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Aug 29 22:29:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:41.071369 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca59df-e155-4a84-a73f-73bc524f7f10 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:41.125727 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0625c6dc-724c-4776-b2aa-a623f1da9beb None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:41.171924 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435db0d8-a001-49c9-add9-abfccabc7218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:41.172973 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435db0d8-a001-49c9-add9-abfccabc7218 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:41.173451 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435db0d8-a001-49c9-add9-abfccabc7218 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:41.174051 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435db0d8-a001-49c9-add9-abfccabc7218 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:41.175383 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435db0d8-a001-49c9-add9-abfccabc7218 None admin] RBAC: Adding query filter params (role.id=882f35bd847a406eb2222aeafc517a46, scope.project.id=ee84c3340f2641ad963655a7a212ef96, user.id=57c175ac90e545358746176972ff39a3) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:41.175892 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435db0d8-a001-49c9-add9-abfccabc7218 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:41.177608 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435db0d8-a001-49c9-add9-abfccabc7218 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:41.184535 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 426/866] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:29:40 2022] GET /identity/v3/role_assignments?user.id=57c175ac90e545358746176972ff39a3&scope.project.id=ee84c3340f2641ad963655a7a212ef96&role.id=882f35bd847a406eb2222aeafc517a46 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:41.201937 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca59df-e155-4a84-a73f-73bc524f7f10 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:41.230970 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:41.243241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097fc1c-5acb-412b-86d3-ad6fa5e141f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:41.244264 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097fc1c-5acb-412b-86d3-ad6fa5e141f8 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:41.244738 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097fc1c-5acb-412b-86d3-ad6fa5e141f8 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:41.254303 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097fc1c-5acb-412b-86d3-ad6fa5e141f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:41.254303 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3097fc1c-5acb-412b-86d3-ad6fa5e141f8 None admin] RBAC: Adding query filter params (role.id=db83c2f8e21945c3ab9642c620d29a23, scope.project.id=ee84c3340f2641ad963655a7a212ef96, user.id=f77dfe78b1a8405084c06d9acee96511) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:41.254303 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3097fc1c-5acb-412b-86d3-ad6fa5e141f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:41.254303 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3097fc1c-5acb-412b-86d3-ad6fa5e141f8 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:41.255094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 441/867] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:29:41 2022] GET /identity/v3/role_assignments?user.id=f77dfe78b1a8405084c06d9acee96511&scope.project.id=ee84c3340f2641ad963655a7a212ef96&role.id=db83c2f8e21945c3ab9642c620d29a23 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:29:41.281199 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435db0d8-a001-49c9-add9-abfccabc7218 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:41.320419 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:41.412142 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435db0d8-a001-49c9-add9-abfccabc7218 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:41.450019 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ec39a-9497-44c8-ae7f-24d388f302f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:41.450648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ec39a-9497-44c8-ae7f-24d388f302f6 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:41.450931 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ec39a-9497-44c8-ae7f-24d388f302f6 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:41.451206 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ec39a-9497-44c8-ae7f-24d388f302f6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:41.451793 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5ec39a-9497-44c8-ae7f-24d388f302f6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:41.453222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5ec39a-9497-44c8-ae7f-24d388f302f6 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:41.463236 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-0d5ec39a-9497-44c8-ae7f-24d388f302f6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 22:29:41.463236 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:41.463236 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:29:41.463236 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:29:41.463236 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:29:41.463236 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:29:41.463236 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:29:41.463236 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:29:41.463236 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 22:29:41.463236 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:41.463236 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:29:41.463236 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:41.463236 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:41.463236 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:41.463236 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:41.463236 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:41.464219 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:41.464219 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:41.464219 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:41.464219 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:41.464219 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:41.464219 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:41.464219 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:41.464219 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:29:41.464219 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:29:41.464219 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:29:41.464219 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:29:41.464219 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:41.464219 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:41.464219 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:41.464219 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:41.464219 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:41.464219 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:41.465755 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:41.465755 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:41.465755 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:41.465755 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:41.465755 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:41.465755 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:41.465755 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:41.465755 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:41.465755 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:41.465755 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:41.465755 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:29:41.465755 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:29:41.465755 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 22:29:41.465755 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:41.465755 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 427/868] 10.222.0.52 () {56 vars in 1132 bytes} [Mon Aug 29 22:29:41 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:41.470145 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097fc1c-5acb-412b-86d3-ad6fa5e141f8 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:41.511995 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c41f94-21a0-40c6-961f-610b371b5b60 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:41.512595 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c41f94-21a0-40c6-961f-610b371b5b60 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:41.513115 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c41f94-21a0-40c6-961f-610b371b5b60 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:41.514987 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 428/869] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:29:41.523749 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2aa6d-2a7d-48a3-8b52-fb86c60ca48c None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:41.524325 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2aa6d-2a7d-48a3-8b52-fb86c60ca48c None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:41.524861 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2aa6d-2a7d-48a3-8b52-fb86c60ca48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:41.546479 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-0ba2aa6d-2a7d-48a3-8b52-fb86c60ca48c None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:41.547843 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-0ba2aa6d-2a7d-48a3-8b52-fb86c60ca48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:41.605488 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097fc1c-5acb-412b-86d3-ad6fa5e141f8 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:41.628468 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 429/870] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:29:41.646987 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe74db-472f-4730-810c-34ad72a2454e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:41.648194 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe74db-472f-4730-810c-34ad72a2454e None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:41.648624 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe74db-472f-4730-810c-34ad72a2454e None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:41.649068 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe74db-472f-4730-810c-34ad72a2454e None admin] PATH_INFO: `/v3/services` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:41.649864 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fe74db-472f-4730-810c-34ad72a2454e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:41.650313 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fe74db-472f-4730-810c-34ad72a2454e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:41.651815 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fe74db-472f-4730-810c-34ad72a2454e None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:41.657058 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 442/871] 10.222.0.52 () {56 vars in 1126 bytes} [Mon Aug 29 22:29:41 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:41.668779 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba2aa6d-2a7d-48a3-8b52-fb86c60ca48c None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:41.683215 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:41.770711 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba2aa6d-2a7d-48a3-8b52-fb86c60ca48c None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:41.807682 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b4f17-48d5-4248-bf14-84095581c4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:41.808290 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b4f17-48d5-4248-bf14-84095581c4d5 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:41.808567 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b4f17-48d5-4248-bf14-84095581c4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:41.808865 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b4f17-48d5-4248-bf14-84095581c4d5 None admin] PATH_INFO: `/v3/services` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:41.809488 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11b4f17-48d5-4248-bf14-84095581c4d5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:41.809784 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11b4f17-48d5-4248-bf14-84095581c4d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:41.811141 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11b4f17-48d5-4248-bf14-84095581c4d5 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:41.816457 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 430/872] 10.222.0.52 () {56 vars in 1126 bytes} [Mon Aug 29 22:29:41 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:41.910317 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe74db-472f-4730-810c-34ad72a2454e None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:41.972470 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:42.022018 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe74db-472f-4730-810c-34ad72a2454e None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:42.058861 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f652aad-6f9f-4b1e-8eda-ef551284f3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:42.059465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f652aad-6f9f-4b1e-8eda-ef551284f3e8 None admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:42.059747 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f652aad-6f9f-4b1e-8eda-ef551284f3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:42.060019 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f652aad-6f9f-4b1e-8eda-ef551284f3e8 None admin] PATH_INFO: `/v3/services` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:42.060680 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f652aad-6f9f-4b1e-8eda-ef551284f3e8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:42.062085 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f652aad-6f9f-4b1e-8eda-ef551284f3e8 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:42.163685 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 443/873] 10.222.0.52 () {60 vars in 1141 bytes} [Mon Aug 29 22:29:41 2022] POST /identity/v3/services => generated 251 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:29:43.219705 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314a1d4-6845-49b8-99a1-2b3457457238 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:43.220423 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314a1d4-6845-49b8-99a1-2b3457457238 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:43.220423 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314a1d4-6845-49b8-99a1-2b3457457238 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:43.222138 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 431/874] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:29:43.227846 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f0736-2d65-45b3-8edb-7a71edc6e283 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:43.228317 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f0736-2d65-45b3-8edb-7a71edc6e283 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:43.228412 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f0736-2d65-45b3-8edb-7a71edc6e283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:43.251148 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-325f0736-2d65-45b3-8edb-7a71edc6e283 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:43.252896 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-325f0736-2d65-45b3-8edb-7a71edc6e283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:43.338646 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 444/875] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:43 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:29:43.426547 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b55f9b-2ebc-47a4-94be-aeac1a76aab2 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:43.426811 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b55f9b-2ebc-47a4-94be-aeac1a76aab2 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:43.427085 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b55f9b-2ebc-47a4-94be-aeac1a76aab2 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:43.428732 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 432/876] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:29:43.434210 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63c430-7b35-4713-8a1f-ca96889e85c4 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:43.434503 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63c430-7b35-4713-8a1f-ca96889e85c4 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:43.434783 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63c430-7b35-4713-8a1f-ca96889e85c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:43.454282 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-0c63c430-7b35-4713-8a1f-ca96889e85c4 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:43.456247 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-0c63c430-7b35-4713-8a1f-ca96889e85c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:43.489124 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:43.532036 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 445/877] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:43 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:29:43.626532 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b55f9b-2ebc-47a4-94be-aeac1a76aab2 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:43.715233 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:43.737952 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b55f9b-2ebc-47a4-94be-aeac1a76aab2 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:43.773678 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5a7435-666f-4c9b-8138-33b7413a0b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:43.774669 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a7435-666f-4c9b-8138-33b7413a0b70 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:43.775163 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a7435-666f-4c9b-8138-33b7413a0b70 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:43.775630 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a7435-666f-4c9b-8138-33b7413a0b70 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:43.776474 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5a7435-666f-4c9b-8138-33b7413a0b70 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:43.778075 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5a7435-666f-4c9b-8138-33b7413a0b70 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:43.788016 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-9b5a7435-666f-4c9b-8138-33b7413a0b70 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 22:29:43.788016 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:43.788016 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:29:43.788016 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:29:43.788016 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:29:43.788016 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:29:43.788016 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:29:43.788016 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:29:43.788016 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 22:29:43.788016 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:43.788016 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:29:43.788016 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:43.788016 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:43.788016 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:43.788016 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:43.788016 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:43.788016 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:43.789070 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:43.789070 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:43.789070 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:43.789070 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:43.789070 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:43.789070 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:43.789070 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:29:43.789070 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:29:43.789070 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:29:43.789070 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:29:43.789070 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:43.789070 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:43.789070 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:43.789070 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:43.789070 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:43.789070 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:43.789890 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:43.789890 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:43.789890 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:43.789890 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:43.789890 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:43.789890 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:43.789890 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:43.789890 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:43.789890 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:43.789890 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:43.789890 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:29:43.789890 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:29:43.789890 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 22:29:43.789890 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:43.792572 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 433/878] 10.222.0.52 () {56 vars in 1126 bytes} [Mon Aug 29 22:29:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:43.800747 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c63c430-7b35-4713-8a1f-ca96889e85c4 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:43.807965 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5a7435-666f-4c9b-8138-33b7413a0b70 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:43.920637 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c63c430-7b35-4713-8a1f-ca96889e85c4 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:43.923127 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5a7435-666f-4c9b-8138-33b7413a0b70 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:43.959474 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b41961-aa27-463b-be6a-b3bba61fd59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:43.960111 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b41961-aa27-463b-be6a-b3bba61fd59f None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:43.961057 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b41961-aa27-463b-be6a-b3bba61fd59f None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:43.961057 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b41961-aa27-463b-be6a-b3bba61fd59f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:43.961218 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5c1764-dd9b-42a9-a107-a8f7d55463d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:43.961787 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c1764-dd9b-42a9-a107-a8f7d55463d0 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:43.962241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c1764-dd9b-42a9-a107-a8f7d55463d0 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:43.962858 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c1764-dd9b-42a9-a107-a8f7d55463d0 None admin] PATH_INFO: `/v3/services` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:43.963956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5c1764-dd9b-42a9-a107-a8f7d55463d0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:43.964437 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5c1764-dd9b-42a9-a107-a8f7d55463d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:43.966298 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5c1764-dd9b-42a9-a107-a8f7d55463d0 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:43.971918 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b41961-aa27-463b-be6a-b3bba61fd59f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:29:43.973288 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 446/879] 10.222.0.52 () {56 vars in 1122 bytes} [Mon Aug 29 22:29:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:43.974634 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d1a48-b69f-499a-9092-d7d83fbec60b None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:43.975046 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d1a48-b69f-499a-9092-d7d83fbec60b None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:43.975444 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d1a48-b69f-499a-9092-d7d83fbec60b None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:43.977008 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 447/880] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:29:43.977599 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-64b41961-aa27-463b-be6a-b3bba61fd59f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 22:29:43.977599 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:43.977599 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:43.977599 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:43.977599 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:43.977599 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:43.977599 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:43.977599 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:43.977599 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:43.977599 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:43.977599 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:43.977599 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:43.977599 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:29:43.977599 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:29:43.977599 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:29:43.977599 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:29:43.978504 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:29:43.978504 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:29:43.978504 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:29:43.978504 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:29:43.978504 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:43.978504 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:43.978504 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:43.978504 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:43.978504 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:43.978504 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:43.978504 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:43.978504 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:43.978504 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:43.978504 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:43.978504 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:43.978504 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:43.983984 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:43.983984 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:43.983984 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:43.983984 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:43.983984 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:29:43.983984 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:29:43.983984 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:29:43.983984 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:29:43.983984 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:29:43.983984 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:29:43.983984 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 22:29:43.983984 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:43.983984 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 434/881] 10.222.0.52 () {56 vars in 1120 bytes} [Mon Aug 29 22:29:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:29:43.984666 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d1a48-b69f-499a-9092-d7d83fbec60b None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:43.984759 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090f7e57-8dab-4e65-bcf3-8bac446faeec None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:43.984759 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090f7e57-8dab-4e65-bcf3-8bac446faeec None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:43.984759 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090f7e57-8dab-4e65-bcf3-8bac446faeec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:43.992609 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:44.005360 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-090f7e57-8dab-4e65-bcf3-8bac446faeec None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:44.006565 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-090f7e57-8dab-4e65-bcf3-8bac446faeec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:44.016603 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:44.076046 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 435/882] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:43 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:29:44.077355 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090f7e57-8dab-4e65-bcf3-8bac446faeec None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:44.104122 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d1a48-b69f-499a-9092-d7d83fbec60b None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:44.145199 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527f3ce-2c6a-4d97-8829-662b0ab9810d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:44.146195 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527f3ce-2c6a-4d97-8829-662b0ab9810d None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:44.146693 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527f3ce-2c6a-4d97-8829-662b0ab9810d None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:44.147223 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527f3ce-2c6a-4d97-8829-662b0ab9810d None admin] PATH_INFO: `/v3/services` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:44.148058 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527f3ce-2c6a-4d97-8829-662b0ab9810d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:44.148579 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527f3ce-2c6a-4d97-8829-662b0ab9810d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:44.150160 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527f3ce-2c6a-4d97-8829-662b0ab9810d None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:44.155487 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 448/883] 10.222.0.52 () {56 vars in 1122 bytes} [Mon Aug 29 22:29:43 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:44.198609 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633bc1c-307a-46c1-8abe-dbdbc62379c3 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:44.198921 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633bc1c-307a-46c1-8abe-dbdbc62379c3 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:44.199152 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633bc1c-307a-46c1-8abe-dbdbc62379c3 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:44.200878 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 449/884] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:29:44.206465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb537b90-6a4f-4138-bd2e-05a4251d4bd9 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:44.206810 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb537b90-6a4f-4138-bd2e-05a4251d4bd9 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:44.207007 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb537b90-6a4f-4138-bd2e-05a4251d4bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:44.207930 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090f7e57-8dab-4e65-bcf3-8bac446faeec None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:44.226133 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-eb537b90-6a4f-4138-bd2e-05a4251d4bd9 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:44.227339 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-eb537b90-6a4f-4138-bd2e-05a4251d4bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:44.252003 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9ce0f3-741d-4bf9-b372-4c8222520e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:44.254244 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9ce0f3-741d-4bf9-b372-4c8222520e29 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:44.254733 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9ce0f3-741d-4bf9-b372-4c8222520e29 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:44.255256 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9ce0f3-741d-4bf9-b372-4c8222520e29 None admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:44.256281 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9ce0f3-741d-4bf9-b372-4c8222520e29 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:44.258749 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9ce0f3-741d-4bf9-b372-4c8222520e29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:44.260562 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9ce0f3-741d-4bf9-b372-4c8222520e29 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:44.262587 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:44.271804 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 436/885] 10.222.0.52 () {56 vars in 1115 bytes} [Mon Aug 29 22:29:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:44.277733 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9ce0f3-741d-4bf9-b372-4c8222520e29 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:44.305785 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 450/886] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:44 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:29:44.357036 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb537b90-6a4f-4138-bd2e-05a4251d4bd9 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:44.393742 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:44.398520 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9ce0f3-741d-4bf9-b372-4c8222520e29 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:44.439083 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149a8cb-8d6a-4121-be4d-bb56b323b20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:44.440756 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149a8cb-8d6a-4121-be4d-bb56b323b20b None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:44.441051 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149a8cb-8d6a-4121-be4d-bb56b323b20b None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:44.441328 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149a8cb-8d6a-4121-be4d-bb56b323b20b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:44.457256 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0149a8cb-8d6a-4121-be4d-bb56b323b20b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:44.459278 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0149a8cb-8d6a-4121-be4d-bb56b323b20b None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:44.478714 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-0149a8cb-8d6a-4121-be4d-bb56b323b20b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:44.478714 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:44.478714 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:44.478714 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:44.478714 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:44.478714 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:44.478714 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:44.478714 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:44.478714 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:44.478714 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:44.478714 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:44.478714 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:44.478714 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:44.478714 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:44.478714 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:44.478714 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:44.479695 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:44.479695 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:44.479695 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:44.479695 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:44.479695 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:44.479695 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:44.479695 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:44.479695 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:44.479695 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:44.479695 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:44.479695 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:44.479695 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:44.479695 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:44.479695 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:44.479695 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:44.479695 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:44.479695 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:44.480707 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:44.480707 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:44.480707 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:44.480707 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 437/887] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:44.488320 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149a8cb-8d6a-4121-be4d-bb56b323b20b None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:44.490518 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb537b90-6a4f-4138-bd2e-05a4251d4bd9 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:44.531759 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece7d5b8-42ff-46a2-936b-84c35b8877a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:44.532376 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece7d5b8-42ff-46a2-936b-84c35b8877a7 None admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:44.532658 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece7d5b8-42ff-46a2-936b-84c35b8877a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:44.540643 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece7d5b8-42ff-46a2-936b-84c35b8877a7 None admin] PATH_INFO: `/v3/services` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:44.540643 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece7d5b8-42ff-46a2-936b-84c35b8877a7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:44.540643 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece7d5b8-42ff-46a2-936b-84c35b8877a7 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:44.567250 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:44.619045 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149a8cb-8d6a-4121-be4d-bb56b323b20b None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:44.649204 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 451/888] 10.222.0.52 () {60 vars in 1141 bytes} [Mon Aug 29 22:29:44 2022] POST /identity/v3/services => generated 269 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:29:44.651762 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece7d5b8-42ff-46a2-936b-84c35b8877a7 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:44.666301 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5335c7d1-8013-444c-9004-c66bed82fcf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:44.667005 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335c7d1-8013-444c-9004-c66bed82fcf0 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:44.667292 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335c7d1-8013-444c-9004-c66bed82fcf0 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:44.667564 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335c7d1-8013-444c-9004-c66bed82fcf0 None admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:44.668231 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5335c7d1-8013-444c-9004-c66bed82fcf0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:44.669398 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5335c7d1-8013-444c-9004-c66bed82fcf0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:44.672414 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5335c7d1-8013-444c-9004-c66bed82fcf0 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:44.692569 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 438/889] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:44.693872 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5335c7d1-8013-444c-9004-c66bed82fcf0 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:44.782903 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:44.804619 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece7d5b8-42ff-46a2-936b-84c35b8877a7 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:44.820624 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5335c7d1-8013-444c-9004-c66bed82fcf0 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:44.843228 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d1db9-a310-4471-8f3a-7d951f4df3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:44.844181 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d1db9-a310-4471-8f3a-7d951f4df3f3 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:44.844658 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d1db9-a310-4471-8f3a-7d951f4df3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:44.845188 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d1db9-a310-4471-8f3a-7d951f4df3f3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:44.846480 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035d1db9-a310-4471-8f3a-7d951f4df3f3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:44.848213 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035d1db9-a310-4471-8f3a-7d951f4df3f3 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:44.860692 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-035d1db9-a310-4471-8f3a-7d951f4df3f3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 22:29:44.860692 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:44.860692 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:29:44.860692 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:29:44.860692 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:29:44.860692 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:29:44.860692 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:29:44.860692 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:29:44.860692 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 22:29:44.860692 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:44.860692 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:29:44.860692 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:44.860692 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:44.860692 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:44.860692 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:44.860692 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:44.860692 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:44.862866 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:44.862866 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:44.862866 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:44.862866 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:44.862866 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:44.862866 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:44.862866 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:29:44.862866 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:29:44.862866 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:29:44.862866 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:29:44.862866 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:44.862866 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:44.862866 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:44.862866 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:44.862866 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:44.862866 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:44.863735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:44.863735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:44.863735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:44.863735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:44.863735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:44.863735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:44.863735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:44.863735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:44.863735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:44.863735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:44.863735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:29:44.863735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:29:44.863735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 22:29:44.863735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:44.864468 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a65dd2-c934-49e1-9d5d-8b741277f624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:44.864468 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a65dd2-c934-49e1-9d5d-8b741277f624 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:44.864468 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a65dd2-c934-49e1-9d5d-8b741277f624 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:44.864672 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a65dd2-c934-49e1-9d5d-8b741277f624 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:44.866335 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 452/890] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Aug 29 22:29:44 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:44.872481 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a65dd2-c934-49e1-9d5d-8b741277f624 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:44.872874 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d1db9-a310-4471-8f3a-7d951f4df3f3 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:44.873927 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a65dd2-c934-49e1-9d5d-8b741277f624 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:44.888037 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-68a65dd2-c934-49e1-9d5d-8b741277f624 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 22:29:44.888037 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:44.888037 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:44.888037 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:44.888037 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:44.888037 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:44.888037 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:44.888037 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:44.888037 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:44.888037 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:44.888037 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:44.888037 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:44.888037 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:29:44.888037 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:29:44.888037 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:29:44.888037 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:29:44.889226 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:44.889226 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:44.889226 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:29:44.889226 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:44.889226 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:29:44.889226 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:44.889226 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:44.889226 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:44.889226 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:44.889226 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:44.889226 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:44.889226 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:44.889226 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:44.889226 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:44.889226 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:44.889226 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:44.889226 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:44.889226 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:44.890379 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:44.890379 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:44.890379 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:29:44.890379 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:29:44.890379 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:29:44.890379 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:29:44.890379 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:29:44.890379 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:29:44.890379 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 22:29:44.890379 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:44.893465 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 439/891] 10.222.0.52 () {56 vars in 1117 bytes} [Mon Aug 29 22:29:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:44.895560 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:44.899335 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a65dd2-c934-49e1-9d5d-8b741277f624 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:45.004790 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d1db9-a310-4471-8f3a-7d951f4df3f3 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:45.009498 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a65dd2-c934-49e1-9d5d-8b741277f624 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:45.047109 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39d8a2b-2ce6-4024-a509-54479a00d03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:45.047826 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d8a2b-2ce6-4024-a509-54479a00d03d None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:45.048185 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d8a2b-2ce6-4024-a509-54479a00d03d None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:45.048488 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d8a2b-2ce6-4024-a509-54479a00d03d None admin] PATH_INFO: `/v3/services` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:45.049275 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39d8a2b-2ce6-4024-a509-54479a00d03d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:45.049661 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39d8a2b-2ce6-4024-a509-54479a00d03d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:45.051361 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39d8a2b-2ce6-4024-a509-54479a00d03d None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:45.056453 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 453/892] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Aug 29 22:29:44 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:45.056597 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5ec651-e7d1-431d-99d4-6cb7da591207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:45.057195 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ec651-e7d1-431d-99d4-6cb7da591207 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:45.057478 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ec651-e7d1-431d-99d4-6cb7da591207 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:45.057752 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ec651-e7d1-431d-99d4-6cb7da591207 None admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:45.058486 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5ec651-e7d1-431d-99d4-6cb7da591207 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:45.058790 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5ec651-e7d1-431d-99d4-6cb7da591207 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:45.060217 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5ec651-e7d1-431d-99d4-6cb7da591207 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:45.063562 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39d8a2b-2ce6-4024-a509-54479a00d03d None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:45.088342 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 440/893] 10.222.0.52 () {56 vars in 1149 bytes} [Mon Aug 29 22:29:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:29:45.090114 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5ec651-e7d1-431d-99d4-6cb7da591207 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:45.125189 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:45.181921 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39d8a2b-2ce6-4024-a509-54479a00d03d None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:45.218572 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5ec651-e7d1-431d-99d4-6cb7da591207 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:45.225238 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:45.228480 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f6295-2bdb-4db9-a53b-067d5348a8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:45.229676 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f6295-2bdb-4db9-a53b-067d5348a8f9 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:45.230156 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f6295-2bdb-4db9-a53b-067d5348a8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:45.230860 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f6295-2bdb-4db9-a53b-067d5348a8f9 None admin] PATH_INFO: `/v3/services` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:45.231929 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803f6295-2bdb-4db9-a53b-067d5348a8f9 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:45.232389 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803f6295-2bdb-4db9-a53b-067d5348a8f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:45.234520 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803f6295-2bdb-4db9-a53b-067d5348a8f9 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:45.246758 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 454/894] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Aug 29 22:29:45 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:45.249071 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f6295-2bdb-4db9-a53b-067d5348a8f9 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:45.258751 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf6df5d-4e4a-4202-8919-8e725811dcbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:45.259428 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf6df5d-4e4a-4202-8919-8e725811dcbb None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:45.259853 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf6df5d-4e4a-4202-8919-8e725811dcbb None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:45.260201 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf6df5d-4e4a-4202-8919-8e725811dcbb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:45.363068 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf6df5d-4e4a-4202-8919-8e725811dcbb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:45.364576 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf6df5d-4e4a-4202-8919-8e725811dcbb None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:45.381738 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:45.395186 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f6295-2bdb-4db9-a53b-067d5348a8f9 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:45.401737 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:45.432295 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36481f68-9375-41e0-9155-8516aed3fa13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:45.433030 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36481f68-9375-41e0-9155-8516aed3fa13 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:45.433401 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36481f68-9375-41e0-9155-8516aed3fa13 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:45.433748 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36481f68-9375-41e0-9155-8516aed3fa13 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:45.434521 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36481f68-9375-41e0-9155-8516aed3fa13 None admin] RBAC: Adding query filter params (interface=public, service_id=bdd9c696285a446b9006226193cb8745, region_id=RegionOne) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:45.434902 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36481f68-9375-41e0-9155-8516aed3fa13 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:45.436339 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36481f68-9375-41e0-9155-8516aed3fa13 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:45.444949 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 455/895] 10.222.0.52 () {56 vars in 1251 bytes} [Mon Aug 29 22:29:45 2022] GET /identity/v3/endpoints?service_id=bdd9c696285a446b9006226193cb8745&interface=public®ion_id=RegionOne => generated 194 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:29:45.450635 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36481f68-9375-41e0-9155-8516aed3fa13 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:45.492816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 441/896] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:29:45 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 22:29:45.496192 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf6df5d-4e4a-4202-8919-8e725811dcbb None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:45.576301 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36481f68-9375-41e0-9155-8516aed3fa13 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:45.615152 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b689715f-2b95-4d36-9410-18f4c7017130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:45.615957 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b689715f-2b95-4d36-9410-18f4c7017130 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:45.616339 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b689715f-2b95-4d36-9410-18f4c7017130 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:45.616712 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b689715f-2b95-4d36-9410-18f4c7017130 None admin] PATH_INFO: `/v3/services` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:45.618012 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b689715f-2b95-4d36-9410-18f4c7017130 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:45.619771 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b689715f-2b95-4d36-9410-18f4c7017130 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:45.628676 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 456/897] 10.222.0.52 () {56 vars in 1087 bytes} [Mon Aug 29 22:29:45 2022] GET /identity/v3/services => generated 1268 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:29:45.635396 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf6df5d-4e4a-4202-8919-8e725811dcbb None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:45.665465 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:45.679462 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438c2cb5-8b58-4b1c-8219-91d0cedf9da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:45.681918 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c2cb5-8b58-4b1c-8219-91d0cedf9da4 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:45.681918 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c2cb5-8b58-4b1c-8219-91d0cedf9da4 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:45.681918 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c2cb5-8b58-4b1c-8219-91d0cedf9da4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:45.693383 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438c2cb5-8b58-4b1c-8219-91d0cedf9da4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:45.695194 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438c2cb5-8b58-4b1c-8219-91d0cedf9da4 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:45.712410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-438c2cb5-8b58-4b1c-8219-91d0cedf9da4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:45.712410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:45.712410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:45.712410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:45.712410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:45.712410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:45.712410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:45.712410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:45.712410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:45.712410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:45.712410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:45.712410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:45.712410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:45.712410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:45.712410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:45.712410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:45.713553 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:45.713553 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:45.713553 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:45.713553 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:45.713553 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:45.713553 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:45.713553 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:45.713553 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:45.713553 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:45.713553 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:45.713553 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:45.713553 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:45.713553 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:45.713553 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:45.713553 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:45.713553 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:45.713553 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:45.714527 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:45.714527 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:45.714527 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:45.716399 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 442/898] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:45.722680 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b689715f-2b95-4d36-9410-18f4c7017130 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:45.742382 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:45.840296 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b689715f-2b95-4d36-9410-18f4c7017130 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:45.879582 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae44b0-c757-45ca-96b2-13ccef14d6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:45.880481 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae44b0-c757-45ca-96b2-13ccef14d6a2 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:45.880952 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae44b0-c757-45ca-96b2-13ccef14d6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:45.881374 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae44b0-c757-45ca-96b2-13ccef14d6a2 None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:45.882165 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ae44b0-c757-45ca-96b2-13ccef14d6a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:45.882646 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ae44b0-c757-45ca-96b2-13ccef14d6a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:45.884243 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ae44b0-c757-45ca-96b2-13ccef14d6a2 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:45.897647 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 457/899] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:45.903385 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438c2cb5-8b58-4b1c-8219-91d0cedf9da4 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:46.023107 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438c2cb5-8b58-4b1c-8219-91d0cedf9da4 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:46.059801 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec1e3c2-e43b-47ec-81e6-7afb9ba2835c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:46.060711 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec1e3c2-e43b-47ec-81e6-7afb9ba2835c None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:46.061177 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec1e3c2-e43b-47ec-81e6-7afb9ba2835c None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:46.061605 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec1e3c2-e43b-47ec-81e6-7afb9ba2835c None admin] PATH_INFO: `/v3/projects/service` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:46.068792 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec1e3c2-e43b-47ec-81e6-7afb9ba2835c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:46.070422 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec1e3c2-e43b-47ec-81e6-7afb9ba2835c None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:46.082424 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-aec1e3c2-e43b-47ec-81e6-7afb9ba2835c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:46.082424 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:46.082424 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:46.082424 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:46.082424 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:46.082424 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:46.082424 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:46.082424 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:46.082424 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:46.082424 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:46.082424 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:46.082424 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:46.082424 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:29:46.082424 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:29:46.082424 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:29:46.083611 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:29:46.083611 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:46.083611 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:46.083611 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:46.083611 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:46.083611 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:46.083611 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:46.083611 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:46.083611 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:46.083611 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:46.083611 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:46.083611 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:46.083611 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:46.083611 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:46.083611 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:46.083611 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:46.083611 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:46.084811 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:29:46.084811 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:29:46.084811 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:29:46.084811 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:29:46.084811 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:29:46.084811 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:29:46.084811 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:46.084811 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:46.087046 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 443/900] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:29:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:46.093165 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae44b0-c757-45ca-96b2-13ccef14d6a2 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:46.169552 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:46.208667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae44b0-c757-45ca-96b2-13ccef14d6a2 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:46.240212 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7e26b-fc25-44aa-b278-9159e7d5f636 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:46.240502 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7e26b-fc25-44aa-b278-9159e7d5f636 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:46.240898 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7e26b-fc25-44aa-b278-9159e7d5f636 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:46.242499 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 444/901] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:29:46.247506 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288a321-f531-4efc-bed2-ddb89c3b588d None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:46.247749 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72365a05-2339-4d99-893e-e504be070aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:46.247879 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288a321-f531-4efc-bed2-ddb89c3b588d None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:46.248154 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288a321-f531-4efc-bed2-ddb89c3b588d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:46.248813 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72365a05-2339-4d99-893e-e504be070aaa None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:46.249248 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72365a05-2339-4d99-893e-e504be070aaa None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:46.249682 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72365a05-2339-4d99-893e-e504be070aaa None admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:46.250513 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72365a05-2339-4d99-893e-e504be070aaa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:46.251078 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72365a05-2339-4d99-893e-e504be070aaa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:46.253756 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72365a05-2339-4d99-893e-e504be070aaa None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:46.269087 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-b288a321-f531-4efc-bed2-ddb89c3b588d None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:46.270567 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-b288a321-f531-4efc-bed2-ddb89c3b588d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:46.275393 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 458/902] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Aug 29 22:29:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:46.281752 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72365a05-2339-4d99-893e-e504be070aaa None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:46.307920 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:46.372754 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 445/903] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:46 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:29:46.403587 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72365a05-2339-4d99-893e-e504be070aaa None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:46.437149 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:46.439202 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf2c08-80ab-442f-8e26-2881110b89df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:46.440380 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf2c08-80ab-442f-8e26-2881110b89df None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:46.440762 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf2c08-80ab-442f-8e26-2881110b89df None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:46.440949 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf2c08-80ab-442f-8e26-2881110b89df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:46.441721 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecf2c08-80ab-442f-8e26-2881110b89df None admin] RBAC: Adding query filter params (role.id=6912999fb1a046b48a32d8fd29c1922d, scope.project.id=ee84c3340f2641ad963655a7a212ef96, user.id=f77dfe78b1a8405084c06d9acee96511) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:46.442022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecf2c08-80ab-442f-8e26-2881110b89df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:46.443489 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecf2c08-80ab-442f-8e26-2881110b89df None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:46.455549 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 459/904] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:29:46 2022] GET /identity/v3/role_assignments?user.id=f77dfe78b1a8405084c06d9acee96511&scope.project.id=ee84c3340f2641ad963655a7a212ef96&role.id=6912999fb1a046b48a32d8fd29c1922d => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:46.651031 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288a321-f531-4efc-bed2-ddb89c3b588d None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:46.739782 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9b9c8-7dee-41f0-8018-af89b657c0ef None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:46.740145 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9b9c8-7dee-41f0-8018-af89b657c0ef None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:46.740344 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9b9c8-7dee-41f0-8018-af89b657c0ef None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:46.741885 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 460/905] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:29:46.747002 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f33e0b9-b30b-4923-ad83-20e66029cb7c None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:46.747404 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f33e0b9-b30b-4923-ad83-20e66029cb7c None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:46.747773 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f33e0b9-b30b-4923-ad83-20e66029cb7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:46.755111 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288a321-f531-4efc-bed2-ddb89c3b588d None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:46.767010 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-7f33e0b9-b30b-4923-ad83-20e66029cb7c None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:46.768436 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-7f33e0b9-b30b-4923-ad83-20e66029cb7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:46.797056 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6ee550-6a04-497a-8558-d19d309a7b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:46.797956 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ee550-6a04-497a-8558-d19d309a7b82 None admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:46.798239 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ee550-6a04-497a-8558-d19d309a7b82 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:46.798509 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ee550-6a04-497a-8558-d19d309a7b82 None admin] PATH_INFO: `/v3/services` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:46.799890 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6ee550-6a04-497a-8558-d19d309a7b82 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:46.801592 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6ee550-6a04-497a-8558-d19d309a7b82 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:46.849359 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 461/906] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:46 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:29:46.897365 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 446/907] 10.222.0.52 () {60 vars in 1141 bytes} [Mon Aug 29 22:29:46 2022] POST /identity/v3/services => generated 241 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:29:47.138205 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f33e0b9-b30b-4923-ad83-20e66029cb7c None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:47.247745 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f33e0b9-b30b-4923-ad83-20e66029cb7c None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:47.270650 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:47.291189 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c39ab1-20ee-4905-83ff-bf5cf846a19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:47.291868 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c39ab1-20ee-4905-83ff-bf5cf846a19d None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:47.292149 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c39ab1-20ee-4905-83ff-bf5cf846a19d None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:47.292423 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c39ab1-20ee-4905-83ff-bf5cf846a19d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:47.293042 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c39ab1-20ee-4905-83ff-bf5cf846a19d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:47.294480 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c39ab1-20ee-4905-83ff-bf5cf846a19d None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:47.305683 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-21c39ab1-20ee-4905-83ff-bf5cf846a19d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 22:29:47.305683 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:47.305683 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:29:47.305683 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:29:47.305683 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:29:47.305683 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:29:47.305683 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:29:47.305683 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:29:47.305683 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 22:29:47.305683 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:47.305683 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:29:47.305683 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:47.305683 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:47.305683 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:47.305683 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:47.305683 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:47.306710 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:47.306710 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:47.306710 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:47.306710 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:47.306710 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:47.306710 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:47.306710 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:47.306710 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:29:47.306710 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:29:47.306710 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:29:47.306710 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:29:47.306710 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:47.306710 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:47.306710 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:47.306710 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:47.306710 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:47.306710 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:47.308025 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:47.308025 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:47.308025 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:47.308025 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:47.308025 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:47.308025 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:47.308025 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:47.308025 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:47.308025 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:47.308025 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:47.308025 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:29:47.308025 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:29:47.308025 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 22:29:47.308025 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:47.308025 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 462/908] 10.222.0.52 () {56 vars in 1132 bytes} [Mon Aug 29 22:29:47 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:47.315741 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6ee550-6a04-497a-8558-d19d309a7b82 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:47.345437 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:47.432304 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6ee550-6a04-497a-8558-d19d309a7b82 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:47.469262 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dfd7a8-5c6b-464f-b533-0c33240a80a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:47.470273 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dfd7a8-5c6b-464f-b533-0c33240a80a0 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:47.470833 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dfd7a8-5c6b-464f-b533-0c33240a80a0 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:47.471796 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dfd7a8-5c6b-464f-b533-0c33240a80a0 None admin] PATH_INFO: `/v3/services` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:47.472632 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dfd7a8-5c6b-464f-b533-0c33240a80a0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:47.473151 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dfd7a8-5c6b-464f-b533-0c33240a80a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:47.474731 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dfd7a8-5c6b-464f-b533-0c33240a80a0 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:47.484845 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 447/909] 10.222.0.52 () {56 vars in 1126 bytes} [Mon Aug 29 22:29:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:47.487015 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c39ab1-20ee-4905-83ff-bf5cf846a19d None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:47.599982 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c39ab1-20ee-4905-83ff-bf5cf846a19d None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:47.640310 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9651c9e5-716a-43bc-a6c0-31461763a9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:47.640953 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651c9e5-716a-43bc-a6c0-31461763a9f8 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:47.641238 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651c9e5-716a-43bc-a6c0-31461763a9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:47.641513 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651c9e5-716a-43bc-a6c0-31461763a9f8 None admin] PATH_INFO: `/v3/services` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:47.642154 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9651c9e5-716a-43bc-a6c0-31461763a9f8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:47.642459 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9651c9e5-716a-43bc-a6c0-31461763a9f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:47.644535 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9651c9e5-716a-43bc-a6c0-31461763a9f8 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:47.650086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 463/910] 10.222.0.52 () {56 vars in 1126 bytes} [Mon Aug 29 22:29:47 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:47.656069 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dfd7a8-5c6b-464f-b533-0c33240a80a0 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:47.720500 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae76abe9-efd9-4d32-9b01-464ca8cd65cd None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:47.720901 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae76abe9-efd9-4d32-9b01-464ca8cd65cd None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:47.721173 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae76abe9-efd9-4d32-9b01-464ca8cd65cd None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:47.722770 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 464/911] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:29:47.728755 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9821c0b9-5ca9-4439-aa98-e255fb8e9ab4 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:47.728755 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9821c0b9-5ca9-4439-aa98-e255fb8e9ab4 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:47.728755 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9821c0b9-5ca9-4439-aa98-e255fb8e9ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:47.747089 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-9821c0b9-5ca9-4439-aa98-e255fb8e9ab4 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:47.748630 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-9821c0b9-5ca9-4439-aa98-e255fb8e9ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:47.751406 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:47.765373 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dfd7a8-5c6b-464f-b533-0c33240a80a0 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:47.797346 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:47.802997 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf05e1fb-c051-47ca-bb2a-53baafac9226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:47.803928 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05e1fb-c051-47ca-bb2a-53baafac9226 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:47.804445 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05e1fb-c051-47ca-bb2a-53baafac9226 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:47.804953 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05e1fb-c051-47ca-bb2a-53baafac9226 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:47.805877 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf05e1fb-c051-47ca-bb2a-53baafac9226 None admin] RBAC: Adding query filter params (interface=public, service_id=ad1ea9d30597466b80f49c32bfbf63d3, region_id=RegionOne) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:47.806413 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf05e1fb-c051-47ca-bb2a-53baafac9226 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:47.808010 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf05e1fb-c051-47ca-bb2a-53baafac9226 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:47.819022 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 448/912] 10.222.0.52 () {56 vars in 1251 bytes} [Mon Aug 29 22:29:47 2022] GET /identity/v3/endpoints?service_id=ad1ea9d30597466b80f49c32bfbf63d3&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:47.820223 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf05e1fb-c051-47ca-bb2a-53baafac9226 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:47.842491 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 465/913] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:29:47.927105 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf05e1fb-c051-47ca-bb2a-53baafac9226 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:47.963191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ee75b-2d73-4639-b690-cd4b0a08cd59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:47.963820 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ee75b-2d73-4639-b690-cd4b0a08cd59 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:47.964082 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ee75b-2d73-4639-b690-cd4b0a08cd59 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:47.967050 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ee75b-2d73-4639-b690-cd4b0a08cd59 None admin] PATH_INFO: `/v3/services` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:47.967698 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8ee75b-2d73-4639-b690-cd4b0a08cd59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:47.969118 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8ee75b-2d73-4639-b690-cd4b0a08cd59 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:47.975782 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 449/914] 10.222.0.52 () {56 vars in 1087 bytes} [Mon Aug 29 22:29:47 2022] GET /identity/v3/services => generated 1497 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:29:48.120863 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9821c0b9-5ca9-4439-aa98-e255fb8e9ab4 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:48.234926 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9821c0b9-5ca9-4439-aa98-e255fb8e9ab4 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:48.271814 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc73ebb1-0224-4c6a-9396-347f9e734ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:48.272886 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc73ebb1-0224-4c6a-9396-347f9e734ee3 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:48.273374 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc73ebb1-0224-4c6a-9396-347f9e734ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:48.273850 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc73ebb1-0224-4c6a-9396-347f9e734ee3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:48.274662 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc73ebb1-0224-4c6a-9396-347f9e734ee3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:48.276240 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc73ebb1-0224-4c6a-9396-347f9e734ee3 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:48.286656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-bc73ebb1-0224-4c6a-9396-347f9e734ee3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 22:29:48.286656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:48.286656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:29:48.286656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:29:48.286656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:29:48.286656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:29:48.286656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:29:48.286656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:29:48.286656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 22:29:48.286656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:48.286656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:29:48.286656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:48.286656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:48.286656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:48.286656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:48.286656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:48.286656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:48.287712 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:48.287712 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:48.287712 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:48.287712 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:48.287712 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:48.287712 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:48.287712 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:29:48.287712 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:29:48.287712 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:29:48.287712 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:29:48.287712 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:48.287712 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:48.287712 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:48.287712 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:48.287712 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:48.287712 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:48.288560 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:48.288560 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:48.288560 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:48.288560 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:48.288560 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:48.288560 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:48.288560 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:48.288560 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:48.288560 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:48.288560 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:48.288560 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:29:48.288560 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:29:48.288560 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 22:29:48.288560 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:48.293094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 466/915] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Aug 29 22:29:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:48.301689 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ee75b-2d73-4639-b690-cd4b0a08cd59 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:48.318786 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:48.414834 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ee75b-2d73-4639-b690-cd4b0a08cd59 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:48.415467 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc945491-180a-444e-b465-921802c28212 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:48.415791 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc945491-180a-444e-b465-921802c28212 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:48.416074 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc945491-180a-444e-b465-921802c28212 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:48.417789 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 467/916] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:29:48.422676 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d2eb3-4497-494a-bc9e-aff64d4e557c None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:48.423229 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d2eb3-4497-494a-bc9e-aff64d4e557c None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:48.423712 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d2eb3-4497-494a-bc9e-aff64d4e557c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:48.443635 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-c36d2eb3-4497-494a-bc9e-aff64d4e557c None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:48.445308 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-c36d2eb3-4497-494a-bc9e-aff64d4e557c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:48.451281 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6c29e-c61a-484b-8d81-34ced79c38ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:48.451887 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6c29e-c61a-484b-8d81-34ced79c38ed None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:48.452162 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6c29e-c61a-484b-8d81-34ced79c38ed None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:48.452431 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6c29e-c61a-484b-8d81-34ced79c38ed None admin] PATH_INFO: `/v3/services` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:48.453096 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f6c29e-c61a-484b-8d81-34ced79c38ed None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:48.453396 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f6c29e-c61a-484b-8d81-34ced79c38ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:48.454907 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f6c29e-c61a-484b-8d81-34ced79c38ed None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:48.460403 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 450/917] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Aug 29 22:29:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:48.466254 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6c29e-c61a-484b-8d81-34ced79c38ed None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:48.491612 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:48.543887 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 468/918] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:29:48.568338 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:48.589777 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6c29e-c61a-484b-8d81-34ced79c38ed None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:48.625935 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3914d41-7863-4bd1-ab79-73d23786e2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:48.627491 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3914d41-7863-4bd1-ab79-73d23786e2bc None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:48.628180 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3914d41-7863-4bd1-ab79-73d23786e2bc None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:48.628669 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3914d41-7863-4bd1-ab79-73d23786e2bc None admin] PATH_INFO: `/v3/services` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:48.629534 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3914d41-7863-4bd1-ab79-73d23786e2bc None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:48.630048 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3914d41-7863-4bd1-ab79-73d23786e2bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:48.631611 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3914d41-7863-4bd1-ab79-73d23786e2bc None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:48.637318 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 451/919] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Aug 29 22:29:48 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:29:48.644080 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36d2eb3-4497-494a-bc9e-aff64d4e557c None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:48.753689 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36d2eb3-4497-494a-bc9e-aff64d4e557c None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:48.789575 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6bbe6-06e1-452c-8277-e2f91506dc01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:48.790182 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6bbe6-06e1-452c-8277-e2f91506dc01 None admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:48.790455 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6bbe6-06e1-452c-8277-e2f91506dc01 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:48.790724 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6bbe6-06e1-452c-8277-e2f91506dc01 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:48.791446 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c6bbe6-06e1-452c-8277-e2f91506dc01 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:48.792872 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c6bbe6-06e1-452c-8277-e2f91506dc01 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:48.817482 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3914d41-7863-4bd1-ab79-73d23786e2bc None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:48.871226 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 469/920] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:29:48 2022] POST /identity/v3/endpoints => generated 346 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:29:48.944921 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3914d41-7863-4bd1-ab79-73d23786e2bc None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:48.990765 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a78d1f1-5c55-482c-9bbc-a7af8a389d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:48.991981 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78d1f1-5c55-482c-9bbc-a7af8a389d8e None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:48.992513 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78d1f1-5c55-482c-9bbc-a7af8a389d8e None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:48.993073 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78d1f1-5c55-482c-9bbc-a7af8a389d8e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:48.994357 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbc993b-480a-4813-9f5c-fa6e89596f0d None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:48.994645 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbc993b-480a-4813-9f5c-fa6e89596f0d None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:48.994925 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbc993b-480a-4813-9f5c-fa6e89596f0d None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:48.996534 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 470/921] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:29:49.002728 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16762cd-1c4a-4c5b-913b-990bd15a072b None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:49.003088 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16762cd-1c4a-4c5b-913b-990bd15a072b None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:49.003319 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16762cd-1c4a-4c5b-913b-990bd15a072b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:49.005529 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:49.010043 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a78d1f1-5c55-482c-9bbc-a7af8a389d8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:29:49.015010 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-5a78d1f1-5c55-482c-9bbc-a7af8a389d8e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 22:29:49.015010 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:49.015010 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:49.015010 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:49.015010 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:49.015010 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:49.015010 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:49.015010 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:49.015010 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:49.015010 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:49.015010 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:49.015010 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:49.015010 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:29:49.015010 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:29:49.015010 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:29:49.015010 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:29:49.019118 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:29:49.019118 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:29:49.019118 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:29:49.019118 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:29:49.019118 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:49.019118 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:49.019118 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:49.019118 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:49.019118 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:49.019118 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:49.019118 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:49.019118 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:49.019118 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:49.019118 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:49.019118 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:49.019118 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:49.019973 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:49.019973 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:49.019973 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:49.019973 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:49.019973 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:29:49.019973 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:29:49.019973 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:29:49.019973 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:29:49.019973 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:29:49.019973 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:29:49.019973 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 22:29:49.019973 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:49.019973 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 452/922] 10.222.0.52 () {56 vars in 1120 bytes} [Mon Aug 29 22:29:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:49.025700 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-d16762cd-1c4a-4c5b-913b-990bd15a072b None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:49.025980 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a78d1f1-5c55-482c-9bbc-a7af8a389d8e None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:49.027042 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-d16762cd-1c4a-4c5b-913b-990bd15a072b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:49.055556 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:49.122744 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 471/923] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:29:49.160902 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a78d1f1-5c55-482c-9bbc-a7af8a389d8e None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:49.197124 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372bb2c-dbfa-48ae-b28c-bee95d78b266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:49.197730 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372bb2c-dbfa-48ae-b28c-bee95d78b266 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:49.198005 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372bb2c-dbfa-48ae-b28c-bee95d78b266 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:49.198277 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372bb2c-dbfa-48ae-b28c-bee95d78b266 None admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:49.198931 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1372bb2c-dbfa-48ae-b28c-bee95d78b266 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:49.199229 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1372bb2c-dbfa-48ae-b28c-bee95d78b266 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:49.200593 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1372bb2c-dbfa-48ae-b28c-bee95d78b266 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:49.209857 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 453/924] 10.222.0.52 () {56 vars in 1115 bytes} [Mon Aug 29 22:29:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:49.215545 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16762cd-1c4a-4c5b-913b-990bd15a072b None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:49.322220 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16762cd-1c4a-4c5b-913b-990bd15a072b None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:49.362794 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adfa374-a16c-4122-aa5a-21e1abb7ea20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:49.363434 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adfa374-a16c-4122-aa5a-21e1abb7ea20 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:49.363715 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adfa374-a16c-4122-aa5a-21e1abb7ea20 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:49.363991 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adfa374-a16c-4122-aa5a-21e1abb7ea20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:49.375553 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adfa374-a16c-4122-aa5a-21e1abb7ea20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:49.377629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adfa374-a16c-4122-aa5a-21e1abb7ea20 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:49.394047 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-2adfa374-a16c-4122-aa5a-21e1abb7ea20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:49.394047 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:49.394047 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:49.394047 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:49.394047 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:49.394047 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:49.394047 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:49.394047 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:49.394047 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:49.394047 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:49.394047 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:49.394047 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:49.394047 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:49.394047 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:49.394047 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:49.394047 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:49.395019 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:49.395019 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:49.395019 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:49.395019 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:49.395019 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:49.395019 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:49.395019 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:49.395019 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:49.395019 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:49.395019 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:49.395019 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:49.395019 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:49.395019 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:49.395019 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:49.395019 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:49.395019 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:49.395019 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:49.396599 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:49.396599 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:49.396599 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:49.396599 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 472/925] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:49.401168 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372bb2c-dbfa-48ae-b28c-bee95d78b266 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:49.407002 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adfa374-a16c-4122-aa5a-21e1abb7ea20 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:49.421805 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:49.486805 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:49.525675 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adfa374-a16c-4122-aa5a-21e1abb7ea20 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:49.532151 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372bb2c-dbfa-48ae-b28c-bee95d78b266 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:49.562877 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29de53d-f599-4b4a-91af-c34fad4863cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:49.563602 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29de53d-f599-4b4a-91af-c34fad4863cf None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:49.563786 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29de53d-f599-4b4a-91af-c34fad4863cf None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:49.564062 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29de53d-f599-4b4a-91af-c34fad4863cf None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:49.565429 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29de53d-f599-4b4a-91af-c34fad4863cf None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:49.566977 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29de53d-f599-4b4a-91af-c34fad4863cf None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:49.573034 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaefb1c-1452-46b5-aa54-7f7c3fc4c2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:49.573859 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaefb1c-1452-46b5-aa54-7f7c3fc4c2b7 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:49.574315 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaefb1c-1452-46b5-aa54-7f7c3fc4c2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:49.574757 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaefb1c-1452-46b5-aa54-7f7c3fc4c2b7 None admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:49.575581 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaefb1c-1452-46b5-aa54-7f7c3fc4c2b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:49.576071 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaefb1c-1452-46b5-aa54-7f7c3fc4c2b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:49.576568 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-a29de53d-f599-4b4a-91af-c34fad4863cf None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 22:29:49.576568 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:49.576568 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:29:49.576568 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:29:49.576568 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:29:49.576568 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:29:49.576568 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:29:49.576568 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:29:49.576568 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 22:29:49.576568 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:49.576568 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:29:49.576568 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:49.576568 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:49.576568 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:49.576568 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:49.576568 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:49.576568 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:49.577569 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:49.577569 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:49.577569 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:49.577569 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:49.577569 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:49.577569 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:49.577569 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:29:49.577569 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:29:49.577569 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:29:49.577569 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:29:49.577569 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:49.577569 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:49.577569 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:49.577569 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:49.577569 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:49.577569 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:49.578407 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:49.578407 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:49.578407 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:49.578407 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:49.578407 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:49.578407 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:49.578407 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:49.578407 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:49.578407 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:49.578407 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:49.578407 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:29:49.578407 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:29:49.578407 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 22:29:49.578407 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:49.581024 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaefb1c-1452-46b5-aa54-7f7c3fc4c2b7 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:49.581149 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 473/926] 10.222.0.52 () {56 vars in 1126 bytes} [Mon Aug 29 22:29:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:29:49.584366 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29de53d-f599-4b4a-91af-c34fad4863cf None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:49.599337 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 454/927] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:49.600953 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaefb1c-1452-46b5-aa54-7f7c3fc4c2b7 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:49.659852 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:49.693695 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29de53d-f599-4b4a-91af-c34fad4863cf None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:49.714072 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaefb1c-1452-46b5-aa54-7f7c3fc4c2b7 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:49.736924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55df98c7-ff31-45b5-beb9-27e0b1c2bed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:49.737715 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df98c7-ff31-45b5-beb9-27e0b1c2bed1 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:49.738109 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df98c7-ff31-45b5-beb9-27e0b1c2bed1 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:49.738416 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df98c7-ff31-45b5-beb9-27e0b1c2bed1 None admin] PATH_INFO: `/v3/services` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:49.739200 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55df98c7-ff31-45b5-beb9-27e0b1c2bed1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:49.739632 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55df98c7-ff31-45b5-beb9-27e0b1c2bed1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:49.741095 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55df98c7-ff31-45b5-beb9-27e0b1c2bed1 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:49.745839 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:49.751954 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 474/928] 10.222.0.52 () {56 vars in 1122 bytes} [Mon Aug 29 22:29:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:49.753770 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55df98c7-ff31-45b5-beb9-27e0b1c2bed1 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:49.759769 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9631070-bd90-43bf-a696-01ca72255d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:49.760672 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9631070-bd90-43bf-a696-01ca72255d15 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:49.761222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9631070-bd90-43bf-a696-01ca72255d15 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:49.761655 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9631070-bd90-43bf-a696-01ca72255d15 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:49.769832 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9631070-bd90-43bf-a696-01ca72255d15 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:49.771560 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9631070-bd90-43bf-a696-01ca72255d15 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:49.786505 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-c9631070-bd90-43bf-a696-01ca72255d15 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 22:29:49.786505 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:49.786505 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:49.786505 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:49.786505 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:49.786505 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:49.786505 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:49.786505 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:49.786505 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:49.786505 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:49.786505 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:49.786505 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:49.786505 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:29:49.786505 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:29:49.786505 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:29:49.786505 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:29:49.787518 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:49.787518 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:49.787518 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:29:49.787518 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:49.787518 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:29:49.787518 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:49.787518 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:49.787518 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:49.787518 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:49.787518 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:49.787518 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:49.787518 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:49.787518 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:49.787518 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:49.787518 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:49.787518 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:49.787518 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:49.787518 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:49.788475 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:49.788475 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:49.788475 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:29:49.788475 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:29:49.788475 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:29:49.788475 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:29:49.788475 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:29:49.788475 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:29:49.788475 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 22:29:49.788475 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:49.795807 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 455/929] 10.222.0.52 () {56 vars in 1117 bytes} [Mon Aug 29 22:29:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:29:49.796990 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9631070-bd90-43bf-a696-01ca72255d15 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:49.865438 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55df98c7-ff31-45b5-beb9-27e0b1c2bed1 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:49.903532 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1a973c-d4df-4553-9fb4-2dd2d05ba14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:49.904391 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a973c-d4df-4553-9fb4-2dd2d05ba14d None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:49.904755 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a973c-d4df-4553-9fb4-2dd2d05ba14d None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:49.905121 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a973c-d4df-4553-9fb4-2dd2d05ba14d None admin] PATH_INFO: `/v3/services` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:49.906139 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1a973c-d4df-4553-9fb4-2dd2d05ba14d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:49.906774 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1a973c-d4df-4553-9fb4-2dd2d05ba14d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:49.908907 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1a973c-d4df-4553-9fb4-2dd2d05ba14d None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:49.909443 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9631070-bd90-43bf-a696-01ca72255d15 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:49.910550 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:49.921038 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 475/930] 10.222.0.52 () {56 vars in 1122 bytes} [Mon Aug 29 22:29:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:49.922004 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1a973c-d4df-4553-9fb4-2dd2d05ba14d None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:49.950377 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032882b6-9720-48be-9bc4-30475fa7d290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:49.951277 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032882b6-9720-48be-9bc4-30475fa7d290 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:49.952651 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032882b6-9720-48be-9bc4-30475fa7d290 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:49.953135 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032882b6-9720-48be-9bc4-30475fa7d290 None admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:49.954040 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032882b6-9720-48be-9bc4-30475fa7d290 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:49.954524 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032882b6-9720-48be-9bc4-30475fa7d290 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:49.956502 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032882b6-9720-48be-9bc4-30475fa7d290 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:49.980618 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 456/931] 10.222.0.52 () {56 vars in 1149 bytes} [Mon Aug 29 22:29:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:49.981879 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032882b6-9720-48be-9bc4-30475fa7d290 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:50.034927 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:50.049792 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1a973c-d4df-4553-9fb4-2dd2d05ba14d None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:50.096466 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6759cd1-8797-478f-8255-7dfb55b4902f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:50.097093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6759cd1-8797-478f-8255-7dfb55b4902f None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:50.097374 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6759cd1-8797-478f-8255-7dfb55b4902f None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:50.097646 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6759cd1-8797-478f-8255-7dfb55b4902f None admin] PATH_INFO: `/v3/endpoints` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:50.098344 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6759cd1-8797-478f-8255-7dfb55b4902f None admin] RBAC: Adding query filter params (interface=public, service_id=ab30c5a3b5084acbbca53743095e472a, region_id=RegionOne) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:50.098651 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6759cd1-8797-478f-8255-7dfb55b4902f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:50.100015 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6759cd1-8797-478f-8255-7dfb55b4902f None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:50.112163 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 476/932] 10.222.0.52 () {56 vars in 1251 bytes} [Mon Aug 29 22:29:49 2022] GET /identity/v3/endpoints?service_id=ab30c5a3b5084acbbca53743095e472a&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:50.112163 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d55561-7d6b-4748-88e7-dc3235a7ebdc None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:50.112163 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d55561-7d6b-4748-88e7-dc3235a7ebdc None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:50.112163 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d55561-7d6b-4748-88e7-dc3235a7ebdc None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:50.112163 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 477/933] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:29:50.114498 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d55561-7d6b-4748-88e7-dc3235a7ebdc None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:50.118526 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032882b6-9720-48be-9bc4-30475fa7d290 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:50.157971 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5b863a-068b-40ae-af5c-6b01bc4ae0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:50.158983 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b863a-068b-40ae-af5c-6b01bc4ae0c0 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:50.159188 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b863a-068b-40ae-af5c-6b01bc4ae0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:50.159427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b863a-068b-40ae-af5c-6b01bc4ae0c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:50.169815 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5b863a-068b-40ae-af5c-6b01bc4ae0c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:50.171293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5b863a-068b-40ae-af5c-6b01bc4ae0c0 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:50.184210 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:50.187042 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-8d5b863a-068b-40ae-af5c-6b01bc4ae0c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:50.187042 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:50.187042 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:50.187042 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:50.187042 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:50.187042 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:50.187042 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:50.187042 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:50.187042 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:50.187042 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:50.187042 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:50.187042 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:50.187042 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:50.187042 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:50.187042 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:50.187042 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:50.188042 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:50.188042 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:50.188042 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:50.188042 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:50.188042 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:50.188042 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:50.188042 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:50.188042 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:50.188042 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:50.188042 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:50.188042 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:50.188042 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:50.188042 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:50.188042 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:50.188042 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:50.188042 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:50.188042 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:50.188960 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:50.188960 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:50.188960 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:50.188960 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 457/934] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:50.190192 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdedb062-5e4e-407b-baa8-ffcec7cbb051 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:50.190192 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdedb062-5e4e-407b-baa8-ffcec7cbb051 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:50.190972 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdedb062-5e4e-407b-baa8-ffcec7cbb051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:50.211330 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-fdedb062-5e4e-407b-baa8-ffcec7cbb051 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:50.212519 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-fdedb062-5e4e-407b-baa8-ffcec7cbb051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:50.233408 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d55561-7d6b-4748-88e7-dc3235a7ebdc None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:50.270865 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52f1d6f-be9a-450e-95e6-9892557dc81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:50.271459 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f1d6f-be9a-450e-95e6-9892557dc81d None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:50.271743 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f1d6f-be9a-450e-95e6-9892557dc81d None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:50.272019 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f1d6f-be9a-450e-95e6-9892557dc81d None admin] PATH_INFO: `/v3/services` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:50.272666 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52f1d6f-be9a-450e-95e6-9892557dc81d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:50.274025 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52f1d6f-be9a-450e-95e6-9892557dc81d None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:50.280544 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 478/935] 10.222.0.52 () {56 vars in 1087 bytes} [Mon Aug 29 22:29:50 2022] GET /identity/v3/services => generated 1497 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:29:50.281428 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52f1d6f-be9a-450e-95e6-9892557dc81d None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:50.304087 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 458/936] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:29:50.418451 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52f1d6f-be9a-450e-95e6-9892557dc81d None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:50.450107 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:50.459793 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76478e2d-b340-4472-97bd-dc46b05fb701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:50.460604 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76478e2d-b340-4472-97bd-dc46b05fb701 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:50.460920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76478e2d-b340-4472-97bd-dc46b05fb701 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:50.461192 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76478e2d-b340-4472-97bd-dc46b05fb701 None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:50.461834 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76478e2d-b340-4472-97bd-dc46b05fb701 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:50.465246 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76478e2d-b340-4472-97bd-dc46b05fb701 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:50.465246 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76478e2d-b340-4472-97bd-dc46b05fb701 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:50.479633 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 479/937] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:50.485226 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdedb062-5e4e-407b-baa8-ffcec7cbb051 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:50.513733 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:50.598255 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdedb062-5e4e-407b-baa8-ffcec7cbb051 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:50.626837 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76478e2d-b340-4472-97bd-dc46b05fb701 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:50.637925 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ba9ab-d615-4e0c-8f7c-6d6dfaaa323d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:50.639251 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ba9ab-d615-4e0c-8f7c-6d6dfaaa323d None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:50.640058 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ba9ab-d615-4e0c-8f7c-6d6dfaaa323d None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:50.640574 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ba9ab-d615-4e0c-8f7c-6d6dfaaa323d None admin] PATH_INFO: `/v3/projects/service` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:50.650611 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316ba9ab-d615-4e0c-8f7c-6d6dfaaa323d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:50.652080 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316ba9ab-d615-4e0c-8f7c-6d6dfaaa323d None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:50.669442 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-316ba9ab-d615-4e0c-8f7c-6d6dfaaa323d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:50.669442 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:50.669442 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:50.669442 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:50.669442 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:50.669442 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:50.669442 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:50.669442 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:50.669442 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:50.669442 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:50.669442 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:50.669442 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:50.669442 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:29:50.669442 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:29:50.669442 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:29:50.670363 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:29:50.670363 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:50.670363 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:50.670363 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:50.670363 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:50.670363 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:50.670363 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:50.670363 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:50.670363 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:50.670363 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:50.670363 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:50.670363 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:50.670363 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:50.670363 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:50.670363 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:50.670363 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:50.670363 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:50.671261 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:29:50.671261 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:29:50.671261 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:29:50.671261 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:29:50.671261 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:29:50.671261 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:29:50.671261 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:50.671261 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:50.671261 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 459/938] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:29:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:50.684500 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ba9ab-d615-4e0c-8f7c-6d6dfaaa323d None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:50.763522 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76478e2d-b340-4472-97bd-dc46b05fb701 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:50.793787 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:50.807833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6046672-a5e7-4ff1-89f7-e25000844fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:50.808628 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ba9ab-d615-4e0c-8f7c-6d6dfaaa323d None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:50.809172 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6046672-a5e7-4ff1-89f7-e25000844fe3 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:50.820998 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6046672-a5e7-4ff1-89f7-e25000844fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:50.820998 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6046672-a5e7-4ff1-89f7-e25000844fe3 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:50.820998 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6046672-a5e7-4ff1-89f7-e25000844fe3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:50.820998 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6046672-a5e7-4ff1-89f7-e25000844fe3 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:50.822955 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-b6046672-a5e7-4ff1-89f7-e25000844fe3 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 22:29:50.822955 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:50.822955 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:29:50.822955 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:29:50.822955 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:29:50.822955 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:29:50.822955 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:29:50.822955 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:29:50.822955 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 22:29:50.822955 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:50.822955 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:29:50.822955 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:50.822955 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:50.822955 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:50.822955 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:50.822955 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:50.823872 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:50.823872 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:50.823872 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:50.823872 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:50.823872 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:50.823872 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:50.823872 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:50.823872 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:29:50.823872 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:29:50.823872 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:29:50.823872 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:29:50.823872 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:50.823872 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:50.823872 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:50.823872 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:50.823872 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:50.823872 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:50.824787 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:50.824787 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:50.824787 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:50.824787 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:50.824787 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:50.824787 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:50.824787 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:50.824787 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:50.824787 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:50.824787 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:50.824787 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:29:50.824787 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:29:50.824787 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 22:29:50.824787 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:50.827512 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 480/939] 10.222.0.52 () {56 vars in 1132 bytes} [Mon Aug 29 22:29:50 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:29:50.836519 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6046672-a5e7-4ff1-89f7-e25000844fe3 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:50.852971 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95151358-3059-47f7-8149-e3a6dd8f78c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:50.853618 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95151358-3059-47f7-8149-e3a6dd8f78c6 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:50.853873 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95151358-3059-47f7-8149-e3a6dd8f78c6 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:50.854148 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95151358-3059-47f7-8149-e3a6dd8f78c6 None admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:50.854841 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95151358-3059-47f7-8149-e3a6dd8f78c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:50.855145 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95151358-3059-47f7-8149-e3a6dd8f78c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:50.856571 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95151358-3059-47f7-8149-e3a6dd8f78c6 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:50.875027 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 460/940] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Aug 29 22:29:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:29:50.875881 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95151358-3059-47f7-8149-e3a6dd8f78c6 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:50.900847 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:50.972507 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6046672-a5e7-4ff1-89f7-e25000844fe3 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:50.991827 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95151358-3059-47f7-8149-e3a6dd8f78c6 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:51.019340 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159970b6-b469-4698-b54c-c4753dd386b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:51.019948 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159970b6-b469-4698-b54c-c4753dd386b9 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:51.020229 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159970b6-b469-4698-b54c-c4753dd386b9 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:51.020503 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159970b6-b469-4698-b54c-c4753dd386b9 None admin] PATH_INFO: `/v3/services` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:51.021164 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159970b6-b469-4698-b54c-c4753dd386b9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:51.021469 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159970b6-b469-4698-b54c-c4753dd386b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:51.022892 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159970b6-b469-4698-b54c-c4753dd386b9 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:51.028818 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 481/941] 10.222.0.52 () {56 vars in 1126 bytes} [Mon Aug 29 22:29:50 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:51.028818 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda56b23-9e81-4819-a23a-1cdaecc1830f None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:51.029037 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda56b23-9e81-4819-a23a-1cdaecc1830f None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:51.029404 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda56b23-9e81-4819-a23a-1cdaecc1830f None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:51.030240 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9633dba4-6ecc-4292-9866-485b21819ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:51.030689 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 482/942] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:29:51.031313 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9633dba4-6ecc-4292-9866-485b21819ef2 None admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:51.031594 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9633dba4-6ecc-4292-9866-485b21819ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:51.031867 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9633dba4-6ecc-4292-9866-485b21819ef2 None admin] PATH_INFO: `/v3/projects/ee84c3340f2641ad963655a7a212ef96/users/f77dfe78b1a8405084c06d9acee96511/roles/6912999fb1a046b48a32d8fd29c1922d` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:51.034342 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda56b23-9e81-4819-a23a-1cdaecc1830f None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:51.034767 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9633dba4-6ecc-4292-9866-485b21819ef2 None admin] RBAC: Authorizing `identity:create_grant(project_id=ee84c3340f2641ad963655a7a212ef96, user_id=f77dfe78b1a8405084c06d9acee96511, role_id=6912999fb1a046b48a32d8fd29c1922d)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:51.036317 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9633dba4-6ecc-4292-9866-485b21819ef2 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:51.139097 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 461/943] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Aug 29 22:29:50 2022] PUT /identity/v3/projects/ee84c3340f2641ad963655a7a212ef96/users/f77dfe78b1a8405084c06d9acee96511/roles/6912999fb1a046b48a32d8fd29c1922d => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:29:51.140773 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab48884-73d8-40f8-a94c-17236877fe7a None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:51.141173 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab48884-73d8-40f8-a94c-17236877fe7a None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:51.141574 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab48884-73d8-40f8-a94c-17236877fe7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:51.150944 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda56b23-9e81-4819-a23a-1cdaecc1830f None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:51.153231 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:51.168214 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:51.170475 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-5ab48884-73d8-40f8-a94c-17236877fe7a None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:51.172409 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-5ab48884-73d8-40f8-a94c-17236877fe7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:51.203152 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beee718d-7911-42ea-a7a0-d09e49f2882e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:51.204470 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee718d-7911-42ea-a7a0-d09e49f2882e None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:51.205081 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee718d-7911-42ea-a7a0-d09e49f2882e None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:51.205946 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee718d-7911-42ea-a7a0-d09e49f2882e None admin] PATH_INFO: `/v3/services` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:51.207737 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beee718d-7911-42ea-a7a0-d09e49f2882e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:51.208436 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beee718d-7911-42ea-a7a0-d09e49f2882e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:51.210719 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beee718d-7911-42ea-a7a0-d09e49f2882e None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:51.223698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 483/944] 10.222.0.52 () {56 vars in 1126 bytes} [Mon Aug 29 22:29:51 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:29:51.225150 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beee718d-7911-42ea-a7a0-d09e49f2882e None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:51.270568 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 462/945] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:29:51.348226 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beee718d-7911-42ea-a7a0-d09e49f2882e None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:51.389827 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7adf29d-c0cd-4530-bf28-997b5ce9f93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:51.390445 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7adf29d-c0cd-4530-bf28-997b5ce9f93d None admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:51.390729 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7adf29d-c0cd-4530-bf28-997b5ce9f93d None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:51.391004 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7adf29d-c0cd-4530-bf28-997b5ce9f93d None admin] PATH_INFO: `/v3/endpoints` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:51.391673 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7adf29d-c0cd-4530-bf28-997b5ce9f93d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:51.393700 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7adf29d-c0cd-4530-bf28-997b5ce9f93d None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:51.518272 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 484/946] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:29:51 2022] POST /identity/v3/endpoints => generated 347 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:29:51.576413 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab48884-73d8-40f8-a94c-17236877fe7a None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:51.688754 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab48884-73d8-40f8-a94c-17236877fe7a None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:51.722091 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:51.732878 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4884804d-8965-42ef-ab79-d611900e9926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:51.733485 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884804d-8965-42ef-ab79-d611900e9926 None admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:51.736984 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884804d-8965-42ef-ab79-d611900e9926 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:51.736984 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884804d-8965-42ef-ab79-d611900e9926 None admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:51.736984 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4884804d-8965-42ef-ab79-d611900e9926 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:51.736984 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4884804d-8965-42ef-ab79-d611900e9926 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:51.740429 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: WARNING keystone.server.flask.common [None req-4884804d-8965-42ef-ab79-d611900e9926 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 29 22:29:51.790476 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.sql.core [None req-4884804d-8965-42ef-ab79-d611900e9926 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 29 22:29:51.790476 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 29 22:29:51.790476 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [parameters: {'id': '56bdc241939a4f138139ea0b3b6be880', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 29 22:29:51.790476 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93272) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 29 22:29:51.808936 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-4884804d-8965-42ef-ab79-d611900e9926 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 56bdc241939a4f138139ea0b3b6be880.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 56bdc241939a4f138139ea0b3b6be880. Aug 29 22:29:51.808936 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:51.808936 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 22:29:51.808936 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 22:29:51.808936 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 22:29:51.808936 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 22:29:51.808936 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 22:29:51.808936 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 22:29:51.808936 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 22:29:51.808936 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application conn.query(q) Aug 29 22:29:51.808936 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 22:29:51.808936 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 22:29:51.808936 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 22:29:51.813781 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application result.read() Aug 29 22:29:51.813781 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 22:29:51.813781 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 22:29:51.813781 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 22:29:51.813781 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 22:29:51.813781 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 22:29:51.813781 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 22:29:51.813781 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 22:29:51.813781 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 22:29:51.813781 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 29 22:29:51.813781 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:51.813781 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 29 22:29:51.813781 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:51.813781 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:51.813781 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 29 22:29:51.813781 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 29 22:29:51.813781 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 29 22:29:51.813781 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 29 22:29:51.815468 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 22:29:51.815468 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application next(self.gen) Aug 29 22:29:51.815468 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 29 22:29:51.815468 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application yield resource Aug 29 22:29:51.815468 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 22:29:51.815468 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application next(self.gen) Aug 29 22:29:51.815468 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 29 22:29:51.815468 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application self.session.rollback() Aug 29 22:29:51.815468 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 29 22:29:51.815468 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application self.force_reraise() Aug 29 22:29:51.815468 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 29 22:29:51.815468 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise self.value Aug 29 22:29:51.815468 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 29 22:29:51.815468 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 29 22:29:51.815468 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 29 22:29:51.815468 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application session.commit() Aug 29 22:29:51.815468 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 29 22:29:51.816348 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 29 22:29:51.816348 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 29 22:29:51.816348 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application self._prepare_impl() Aug 29 22:29:51.816348 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 29 22:29:51.816348 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application self.session.flush() Aug 29 22:29:51.816348 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 29 22:29:51.816348 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application self._flush(objects) Aug 29 22:29:51.816348 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 29 22:29:51.816348 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 29 22:29:51.816348 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 29 22:29:51.816348 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application compat.raise_( Aug 29 22:29:51.816348 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 22:29:51.816348 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception Aug 29 22:29:51.816348 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 29 22:29:51.816348 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application flush_context.execute() Aug 29 22:29:51.816348 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 29 22:29:51.816348 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rec.execute(self) Aug 29 22:29:51.818563 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 29 22:29:51.818563 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 29 22:29:51.818563 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 29 22:29:51.818563 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 29 22:29:51.818563 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 29 22:29:51.818563 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 29 22:29:51.818563 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 29 22:29:51.818563 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 29 22:29:51.818563 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 29 22:29:51.818563 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 29 22:29:51.818563 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 29 22:29:51.818563 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 29 22:29:51.818563 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 29 22:29:51.818563 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 29 22:29:51.818563 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 29 22:29:51.820427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 29 22:29:51.820427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 22:29:51.820427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception Aug 29 22:29:51.820427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 22:29:51.820427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 22:29:51.820427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 22:29:51.820427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 22:29:51.820427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 22:29:51.820427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 22:29:51.820427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 22:29:51.820427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application conn.query(q) Aug 29 22:29:51.820427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 22:29:51.820427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 22:29:51.820427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 22:29:51.820427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application result.read() Aug 29 22:29:51.820427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 22:29:51.820427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 22:29:51.822557 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 22:29:51.822557 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 22:29:51.822557 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 22:29:51.822557 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 22:29:51.822557 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 22:29:51.822557 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 22:29:51.822557 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 29 22:29:51.822557 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 29 22:29:51.822557 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [parameters: {'id': '56bdc241939a4f138139ea0b3b6be880', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 29 22:29:51.822557 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 29 22:29:51.822557 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:51.822557 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:29:51.822557 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:51.822557 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:51.822557 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 29 22:29:51.824425 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 29 22:29:51.824425 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 29 22:29:51.824425 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 29 22:29:51.824425 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 29 22:29:51.824425 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:51.824425 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:29:51.824425 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:51.824425 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:51.824425 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:51.824425 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:51.824425 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:51.824425 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:51.824425 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:51.824425 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:51.824425 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:51.824425 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:51.824425 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:51.830677 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:51.830677 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 29 22:29:51.830677 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 29 22:29:51.830677 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:51.830677 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:51.830677 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 29 22:29:51.830677 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 29 22:29:51.830677 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 56bdc241939a4f138139ea0b3b6be880. Aug 29 22:29:51.830677 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:51.830677 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 463/947] 10.222.0.52 () {60 vars in 1140 bytes} [Mon Aug 29 22:29:51 2022] POST /identity/v3/projects => generated 262 bytes in 235 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:51.832424 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7adf29d-c0cd-4530-bf28-997b5ce9f93d None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:51.869813 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:51.941585 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7adf29d-c0cd-4530-bf28-997b5ce9f93d None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:51.977634 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b3fd4f-04b9-41a0-a428-de9182b3fc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:51.978343 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b3fd4f-04b9-41a0-a428-de9182b3fc38 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:51.978784 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b3fd4f-04b9-41a0-a428-de9182b3fc38 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:51.979059 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b3fd4f-04b9-41a0-a428-de9182b3fc38 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:51.986107 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b3fd4f-04b9-41a0-a428-de9182b3fc38 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:51.987664 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b3fd4f-04b9-41a0-a428-de9182b3fc38 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:51.999255 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-e2b3fd4f-04b9-41a0-a428-de9182b3fc38 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:51.999255 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:51.999255 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:51.999255 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:51.999255 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:51.999255 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:51.999255 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:51.999255 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:51.999255 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:51.999255 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:51.999255 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:51.999255 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:51.999255 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:29:51.999255 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:29:51.999255 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:29:52.000192 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:29:52.000192 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:52.000192 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:52.000192 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:52.000192 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:52.000192 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:52.000192 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:52.000192 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:52.000192 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:52.000192 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:52.000192 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:52.000192 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:52.000192 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:52.000192 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:52.000192 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:52.000192 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:52.000192 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:52.001126 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:29:52.001126 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:29:52.001126 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:29:52.001126 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:29:52.001126 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:29:52.001126 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:29:52.001126 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:52.001126 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:52.001126 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 485/948] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:29:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:52.008564 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4884804d-8965-42ef-ab79-d611900e9926 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:52.119951 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4884804d-8965-42ef-ab79-d611900e9926 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:52.156320 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9b1794-9dd2-4456-86bf-c4446917b521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:52.157009 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b1794-9dd2-4456-86bf-c4446917b521 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:52.157290 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b1794-9dd2-4456-86bf-c4446917b521 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:52.157562 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b1794-9dd2-4456-86bf-c4446917b521 None admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:52.158335 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9b1794-9dd2-4456-86bf-c4446917b521 None admin] RBAC: Adding query filter params (name=service) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:52.159110 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9b1794-9dd2-4456-86bf-c4446917b521 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:52.160751 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9b1794-9dd2-4456-86bf-c4446917b521 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:52.182815 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 464/949] 10.222.0.52 () {56 vars in 1112 bytes} [Mon Aug 29 22:29:52 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:52.355117 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17da67-c753-442e-a956-40eb68aa7f78 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:52.355423 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17da67-c753-442e-a956-40eb68aa7f78 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:52.355676 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17da67-c753-442e-a956-40eb68aa7f78 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:52.357292 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 486/950] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:29:52.364684 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86162156-4f68-4068-85d0-0f762342b386 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:52.365218 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86162156-4f68-4068-85d0-0f762342b386 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:52.365667 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86162156-4f68-4068-85d0-0f762342b386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:52.386158 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-86162156-4f68-4068-85d0-0f762342b386 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:52.387874 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-86162156-4f68-4068-85d0-0f762342b386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:52.435280 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:52.463028 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 465/951] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:29:52.751614 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17da67-c753-442e-a956-40eb68aa7f78 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:52.852263 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:52.859826 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17da67-c753-442e-a956-40eb68aa7f78 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:52.895520 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec13c8-4882-4466-ae17-82dc515a07e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:52.896145 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec13c8-4882-4466-ae17-82dc515a07e7 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:52.896424 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec13c8-4882-4466-ae17-82dc515a07e7 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:52.896696 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec13c8-4882-4466-ae17-82dc515a07e7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:52.897303 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ec13c8-4882-4466-ae17-82dc515a07e7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:52.898658 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ec13c8-4882-4466-ae17-82dc515a07e7 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:52.907742 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-d8ec13c8-4882-4466-ae17-82dc515a07e7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 22:29:52.907742 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:52.907742 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:29:52.907742 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:29:52.907742 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:29:52.907742 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:29:52.907742 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:29:52.907742 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:29:52.907742 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 22:29:52.907742 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:52.907742 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:29:52.907742 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:52.907742 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:52.907742 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:52.907742 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:52.907742 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:52.907742 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:52.909515 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:52.909515 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:52.909515 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:52.909515 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:52.909515 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:52.909515 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:52.909515 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:29:52.909515 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:29:52.909515 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:29:52.909515 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:29:52.909515 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:52.909515 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:52.909515 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:52.909515 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:52.909515 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:52.909515 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:52.910594 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:52.910594 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:52.910594 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:52.910594 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:52.910594 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:52.910594 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:52.910594 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:52.910594 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:52.910594 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:52.910594 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:52.910594 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:29:52.910594 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:29:52.910594 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 22:29:52.910594 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:52.910594 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 487/952] 10.222.0.52 () {56 vars in 1126 bytes} [Mon Aug 29 22:29:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:52.915769 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86162156-4f68-4068-85d0-0f762342b386 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:53.025382 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86162156-4f68-4068-85d0-0f762342b386 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:53.062622 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364ef01f-c14f-472f-a14d-563bb21f6b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:53.063218 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ef01f-c14f-472f-a14d-563bb21f6b95 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:53.063498 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ef01f-c14f-472f-a14d-563bb21f6b95 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:53.063770 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ef01f-c14f-472f-a14d-563bb21f6b95 None admin] PATH_INFO: `/v3/services` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:53.064472 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364ef01f-c14f-472f-a14d-563bb21f6b95 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:53.065324 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364ef01f-c14f-472f-a14d-563bb21f6b95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:53.066721 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364ef01f-c14f-472f-a14d-563bb21f6b95 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:53.072126 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 466/953] 10.222.0.52 () {56 vars in 1122 bytes} [Mon Aug 29 22:29:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:53.077506 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec13c8-4882-4466-ae17-82dc515a07e7 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:53.186374 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec13c8-4882-4466-ae17-82dc515a07e7 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:53.186723 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfff871-b527-4cba-8dfb-cb01a5dd9188 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:53.186886 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfff871-b527-4cba-8dfb-cb01a5dd9188 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:53.187073 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfff871-b527-4cba-8dfb-cb01a5dd9188 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:53.188688 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 467/954] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:29:53.194067 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744f025-2089-4fc8-aa89-717bc6912086 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:53.194607 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744f025-2089-4fc8-aa89-717bc6912086 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:53.195092 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744f025-2089-4fc8-aa89-717bc6912086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:53.218979 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-6744f025-2089-4fc8-aa89-717bc6912086 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:53.221892 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-6744f025-2089-4fc8-aa89-717bc6912086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:53.224898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50daa8aa-1ffb-4654-83da-9f055c26f6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:53.225508 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50daa8aa-1ffb-4654-83da-9f055c26f6f2 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:53.225789 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50daa8aa-1ffb-4654-83da-9f055c26f6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:53.226060 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50daa8aa-1ffb-4654-83da-9f055c26f6f2 None admin] PATH_INFO: `/v3/services` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:53.226692 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50daa8aa-1ffb-4654-83da-9f055c26f6f2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:53.227057 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50daa8aa-1ffb-4654-83da-9f055c26f6f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:53.228482 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50daa8aa-1ffb-4654-83da-9f055c26f6f2 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:53.234046 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 488/955] 10.222.0.52 () {56 vars in 1122 bytes} [Mon Aug 29 22:29:53 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:53.239656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50daa8aa-1ffb-4654-83da-9f055c26f6f2 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:53.261757 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:53.267459 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:53.320707 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 468/956] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:29:53.354573 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50daa8aa-1ffb-4654-83da-9f055c26f6f2 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:53.396246 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b6f0e1-8f59-480a-9922-31adbd38062e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:53.396890 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6f0e1-8f59-480a-9922-31adbd38062e None admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:53.397174 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6f0e1-8f59-480a-9922-31adbd38062e None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:53.397449 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6f0e1-8f59-480a-9922-31adbd38062e None admin] PATH_INFO: `/v3/endpoints` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:53.398117 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b6f0e1-8f59-480a-9922-31adbd38062e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:53.399549 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b6f0e1-8f59-480a-9922-31adbd38062e None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:53.500429 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 489/957] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:29:53 2022] POST /identity/v3/endpoints => generated 349 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:29:53.622892 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eeb1ac-57c7-434b-ae9e-c731830b5580 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:53.623092 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eeb1ac-57c7-434b-ae9e-c731830b5580 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:53.623353 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eeb1ac-57c7-434b-ae9e-c731830b5580 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:53.625036 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 469/958] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:29:53.631354 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0623d6-077d-45c2-b009-598691320684 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:53.631666 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0623d6-077d-45c2-b009-598691320684 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:53.631946 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0623d6-077d-45c2-b009-598691320684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:53.632472 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b6f0e1-8f59-480a-9922-31adbd38062e None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:53.655657 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-bd0623d6-077d-45c2-b009-598691320684 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:53.657306 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-bd0623d6-077d-45c2-b009-598691320684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:53.729879 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:53.757149 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 470/959] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:29:53.762226 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b6f0e1-8f59-480a-9922-31adbd38062e None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:53.798278 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad406b5d-61f7-4268-a1cb-4667cd0926b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:53.799070 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad406b5d-61f7-4268-a1cb-4667cd0926b2 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:53.801088 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad406b5d-61f7-4268-a1cb-4667cd0926b2 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:53.801088 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad406b5d-61f7-4268-a1cb-4667cd0926b2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:53.811493 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad406b5d-61f7-4268-a1cb-4667cd0926b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:29:53.817151 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-ad406b5d-61f7-4268-a1cb-4667cd0926b2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 22:29:53.817151 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:53.817151 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:53.817151 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:53.817151 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:53.817151 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:53.817151 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:53.817151 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:53.817151 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:53.817151 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:53.817151 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:53.817151 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:53.817151 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:29:53.817151 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:29:53.817151 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:29:53.817151 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:29:53.818170 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:29:53.818170 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:29:53.818170 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:29:53.818170 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:29:53.818170 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:53.818170 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:53.818170 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:53.818170 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:53.818170 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:53.818170 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:53.818170 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:53.818170 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:53.818170 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:53.818170 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:53.818170 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:53.818170 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:53.819523 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:53.819523 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:53.819523 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:53.819523 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:53.819523 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:29:53.819523 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:29:53.819523 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:29:53.819523 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:29:53.819523 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:29:53.819523 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:29:53.819523 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 22:29:53.819523 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:53.819523 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 490/960] 10.222.0.52 () {56 vars in 1120 bytes} [Mon Aug 29 22:29:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:53.827515 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0623d6-077d-45c2-b009-598691320684 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:53.934931 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:53.942478 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0623d6-077d-45c2-b009-598691320684 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:53.979384 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988747bf-ca30-477b-a2cc-35f8a62ad35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:53.979998 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988747bf-ca30-477b-a2cc-35f8a62ad35f None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:53.980280 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988747bf-ca30-477b-a2cc-35f8a62ad35f None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:53.980556 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988747bf-ca30-477b-a2cc-35f8a62ad35f None admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:53.981261 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988747bf-ca30-477b-a2cc-35f8a62ad35f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:53.981562 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988747bf-ca30-477b-a2cc-35f8a62ad35f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:53.983000 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988747bf-ca30-477b-a2cc-35f8a62ad35f None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:53.992393 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 471/961] 10.222.0.52 () {56 vars in 1115 bytes} [Mon Aug 29 22:29:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:54.002144 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad406b5d-61f7-4268-a1cb-4667cd0926b2 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:54.080186 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988747bf-ca30-477b-a2cc-35f8a62ad35f None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:54.121198 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad406b5d-61f7-4268-a1cb-4667cd0926b2 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:54.180975 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4561a62-997a-41dc-abdf-eafcf5e17184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:54.182632 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4561a62-997a-41dc-abdf-eafcf5e17184 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:54.183543 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4561a62-997a-41dc-abdf-eafcf5e17184 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:54.184260 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4561a62-997a-41dc-abdf-eafcf5e17184 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:54.188489 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:54.206827 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4561a62-997a-41dc-abdf-eafcf5e17184 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:54.210103 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4561a62-997a-41dc-abdf-eafcf5e17184 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:54.239201 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-d4561a62-997a-41dc-abdf-eafcf5e17184 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:54.239201 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:54.239201 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:54.239201 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:54.239201 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:54.239201 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:54.239201 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:54.239201 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:54.239201 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:54.239201 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:54.239201 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:54.239201 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:54.239201 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:54.239201 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:54.239201 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:54.239201 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:54.241086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:54.241086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:54.241086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:54.241086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:54.241086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:54.241086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:54.241086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:54.241086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:54.241086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:54.241086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:54.241086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:54.241086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:54.241086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:54.241086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:54.241086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:54.241086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:54.241086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:54.242901 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:54.242901 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:54.242901 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:54.246318 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 491/962] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:54.256995 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4561a62-997a-41dc-abdf-eafcf5e17184 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:54.260851 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988747bf-ca30-477b-a2cc-35f8a62ad35f None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:54.323358 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc52767-51dd-472d-b77f-6bf8c749f7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:54.324164 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc52767-51dd-472d-b77f-6bf8c749f7e0 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:54.324562 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc52767-51dd-472d-b77f-6bf8c749f7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:54.324999 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc52767-51dd-472d-b77f-6bf8c749f7e0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:54.325742 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc52767-51dd-472d-b77f-6bf8c749f7e0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:54.327536 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc52767-51dd-472d-b77f-6bf8c749f7e0 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:54.341792 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-3bc52767-51dd-472d-b77f-6bf8c749f7e0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 22:29:54.341792 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:54.341792 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:29:54.341792 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:29:54.341792 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:29:54.341792 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:29:54.341792 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:29:54.341792 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:29:54.341792 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 22:29:54.341792 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:54.341792 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:29:54.341792 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:54.341792 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:54.341792 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:54.341792 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:54.341792 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:54.341792 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:54.342957 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:54.342957 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:54.342957 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:54.342957 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:54.342957 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:54.342957 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:54.342957 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:29:54.342957 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:29:54.342957 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:29:54.342957 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:29:54.342957 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:54.342957 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:54.342957 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:54.342957 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:54.342957 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:54.342957 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:54.343862 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:54.343862 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:54.343862 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:54.343862 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:54.343862 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:54.343862 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:54.343862 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:54.343862 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:54.343862 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:54.343862 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:54.343862 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:29:54.343862 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:29:54.343862 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 22:29:54.343862 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:54.343862 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 472/963] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:29:54 2022] GET /identity/v3/services/compute => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:29:54.346345 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328cfe44-2bfe-4dae-8055-3117ce2e7fa6 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:54.346822 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328cfe44-2bfe-4dae-8055-3117ce2e7fa6 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:54.347283 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328cfe44-2bfe-4dae-8055-3117ce2e7fa6 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:54.353671 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 473/964] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:29:54.353671 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328cfe44-2bfe-4dae-8055-3117ce2e7fa6 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:54.388469 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:54.423637 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4561a62-997a-41dc-abdf-eafcf5e17184 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:54.462312 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee016dfe-4f72-470d-9780-5648324532b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:54.462923 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee016dfe-4f72-470d-9780-5648324532b7 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:54.463207 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee016dfe-4f72-470d-9780-5648324532b7 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:54.463532 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee016dfe-4f72-470d-9780-5648324532b7 None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:54.464284 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee016dfe-4f72-470d-9780-5648324532b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:54.464752 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee016dfe-4f72-470d-9780-5648324532b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:54.466119 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee016dfe-4f72-470d-9780-5648324532b7 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:54.480174 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 492/965] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:54.482038 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d638f90-2884-4f5f-9468-f7d3992b917f None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:54.482310 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328cfe44-2bfe-4dae-8055-3117ce2e7fa6 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:54.482756 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d638f90-2884-4f5f-9468-f7d3992b917f None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:54.483242 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d638f90-2884-4f5f-9468-f7d3992b917f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:54.503400 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-3d638f90-2884-4f5f-9468-f7d3992b917f None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:54.504669 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-3d638f90-2884-4f5f-9468-f7d3992b917f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:54.522483 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ab7c0-326f-4917-9f0e-ea6ae27af73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:54.523089 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ab7c0-326f-4917-9f0e-ea6ae27af73c None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:54.523371 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ab7c0-326f-4917-9f0e-ea6ae27af73c None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:54.523644 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ab7c0-326f-4917-9f0e-ea6ae27af73c None admin] PATH_INFO: `/v3/services` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:54.524336 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467ab7c0-326f-4917-9f0e-ea6ae27af73c None admin] RBAC: Adding query filter params (name=compute) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:54.524694 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467ab7c0-326f-4917-9f0e-ea6ae27af73c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:54.526066 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467ab7c0-326f-4917-9f0e-ea6ae27af73c None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:54.532182 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 474/966] 10.222.0.52 () {56 vars in 1112 bytes} [Mon Aug 29 22:29:54 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:54.538777 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ab7c0-326f-4917-9f0e-ea6ae27af73c None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:54.558013 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:54.585145 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 493/967] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:29:54.586395 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d638f90-2884-4f5f-9468-f7d3992b917f None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:54.665183 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:54.673624 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ab7c0-326f-4917-9f0e-ea6ae27af73c None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:54.718379 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193fb08-ee24-4cd6-8c6b-1fe1d568b390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:54.718929 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193fb08-ee24-4cd6-8c6b-1fe1d568b390 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:54.719210 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193fb08-ee24-4cd6-8c6b-1fe1d568b390 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:54.719490 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193fb08-ee24-4cd6-8c6b-1fe1d568b390 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:54.725581 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d638f90-2884-4f5f-9468-f7d3992b917f None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:54.727658 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193fb08-ee24-4cd6-8c6b-1fe1d568b390 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:54.729154 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193fb08-ee24-4cd6-8c6b-1fe1d568b390 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:54.741639 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-8193fb08-ee24-4cd6-8c6b-1fe1d568b390 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 22:29:54.741639 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:54.741639 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:54.741639 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:54.741639 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:54.741639 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:54.741639 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:54.741639 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:54.741639 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:54.741639 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:54.741639 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:54.741639 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:54.741639 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:29:54.741639 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:29:54.741639 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:29:54.741639 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:29:54.742616 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:54.742616 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:54.742616 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:29:54.742616 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:54.742616 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:29:54.742616 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:54.742616 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:54.742616 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:54.742616 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:54.742616 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:54.742616 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:54.742616 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:54.742616 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:54.742616 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:54.742616 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:54.742616 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:54.742616 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:54.742616 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:54.743635 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:54.743635 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:54.743635 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:29:54.743635 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:29:54.743635 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:29:54.743635 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:29:54.743635 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:29:54.743635 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:29:54.743635 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 22:29:54.743635 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:54.743635 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 475/968] 10.222.0.52 () {56 vars in 1117 bytes} [Mon Aug 29 22:29:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:54.748351 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193fb08-ee24-4cd6-8c6b-1fe1d568b390 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:54.764219 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e345755-d894-4a55-8b42-d56166b036de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:54.764914 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e345755-d894-4a55-8b42-d56166b036de None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:54.765196 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e345755-d894-4a55-8b42-d56166b036de None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:54.765522 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e345755-d894-4a55-8b42-d56166b036de None admin] PATH_INFO: `/v3/services` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:54.766214 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e345755-d894-4a55-8b42-d56166b036de None admin] RBAC: Adding query filter params (type=compute) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:54.766517 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e345755-d894-4a55-8b42-d56166b036de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:54.769028 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e345755-d894-4a55-8b42-d56166b036de None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:54.774074 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 494/969] 10.222.0.52 () {56 vars in 1112 bytes} [Mon Aug 29 22:29:54 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:54.870457 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193fb08-ee24-4cd6-8c6b-1fe1d568b390 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:54.880140 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e345755-d894-4a55-8b42-d56166b036de None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:54.912810 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a96b16-17dc-43ed-99cc-d41e0cb47f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:54.913482 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a96b16-17dc-43ed-99cc-d41e0cb47f5f None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:54.913813 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a96b16-17dc-43ed-99cc-d41e0cb47f5f None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:54.914086 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a96b16-17dc-43ed-99cc-d41e0cb47f5f None admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:54.914942 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a96b16-17dc-43ed-99cc-d41e0cb47f5f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:54.915253 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a96b16-17dc-43ed-99cc-d41e0cb47f5f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:54.916757 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a96b16-17dc-43ed-99cc-d41e0cb47f5f None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:54.936555 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:54.938713 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 476/970] 10.222.0.52 () {56 vars in 1149 bytes} [Mon Aug 29 22:29:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:29:54.944698 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a96b16-17dc-43ed-99cc-d41e0cb47f5f None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:54.947690 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:55.015199 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e345755-d894-4a55-8b42-d56166b036de None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:55.061580 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454e934-f8c7-40dc-8be8-d14c94548371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:55.062259 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454e934-f8c7-40dc-8be8-d14c94548371 None admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:55.062544 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454e934-f8c7-40dc-8be8-d14c94548371 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:55.062816 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454e934-f8c7-40dc-8be8-d14c94548371 None admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:55.063122 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a96b16-17dc-43ed-99cc-d41e0cb47f5f None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:55.063459 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2454e934-f8c7-40dc-8be8-d14c94548371 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:55.065629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2454e934-f8c7-40dc-8be8-d14c94548371 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:55.069124 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: WARNING keystone.server.flask.common [None req-2454e934-f8c7-40dc-8be8-d14c94548371 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 29 22:29:55.102701 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa75305-3f7a-4aa3-aadc-5958fa8b3ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:55.103352 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa75305-3f7a-4aa3-aadc-5958fa8b3ec8 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:55.103629 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa75305-3f7a-4aa3-aadc-5958fa8b3ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:55.103902 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa75305-3f7a-4aa3-aadc-5958fa8b3ec8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:55.116557 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.sql.core [None req-2454e934-f8c7-40dc-8be8-d14c94548371 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 29 22:29:55.116557 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 29 22:29:55.116557 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [parameters: {'user_id': 'c49b5f7a1dc24e77acae39db931e309d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 29 22:29:55.116557 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93271) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 29 22:29:55.126655 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-2454e934-f8c7-40dc-8be8-d14c94548371 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 29 22:29:55.126655 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:55.126655 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 22:29:55.126655 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 22:29:55.126655 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 22:29:55.126655 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 22:29:55.126655 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 22:29:55.126655 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 22:29:55.126655 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 22:29:55.126655 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application conn.query(q) Aug 29 22:29:55.126655 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 22:29:55.126655 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 22:29:55.126655 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 22:29:55.126655 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application result.read() Aug 29 22:29:55.126655 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 22:29:55.128191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 22:29:55.128191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 22:29:55.128191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 22:29:55.128191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 22:29:55.128191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 22:29:55.128191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 22:29:55.128191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 22:29:55.128191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 29 22:29:55.128191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:55.128191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 29 22:29:55.128191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:55.128191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:55.128191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 29 22:29:55.128191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 29 22:29:55.128191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 29 22:29:55.128191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 29 22:29:55.128191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 22:29:55.128191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application next(self.gen) Aug 29 22:29:55.129309 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 29 22:29:55.129309 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application yield resource Aug 29 22:29:55.129309 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 22:29:55.129309 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application next(self.gen) Aug 29 22:29:55.129309 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 29 22:29:55.129309 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application self.session.rollback() Aug 29 22:29:55.129309 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 29 22:29:55.129309 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application self.force_reraise() Aug 29 22:29:55.129309 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 29 22:29:55.129309 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise self.value Aug 29 22:29:55.129309 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 29 22:29:55.129309 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 29 22:29:55.129309 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 29 22:29:55.129309 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application session.commit() Aug 29 22:29:55.129309 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 29 22:29:55.129309 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 29 22:29:55.129309 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 29 22:29:55.130218 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application self._prepare_impl() Aug 29 22:29:55.130218 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 29 22:29:55.130218 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application self.session.flush() Aug 29 22:29:55.130218 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 29 22:29:55.130218 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application self._flush(objects) Aug 29 22:29:55.130218 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 29 22:29:55.130218 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 29 22:29:55.130218 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 29 22:29:55.130218 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application compat.raise_( Aug 29 22:29:55.130218 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 22:29:55.130218 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception Aug 29 22:29:55.130218 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 29 22:29:55.130218 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application flush_context.execute() Aug 29 22:29:55.130218 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 29 22:29:55.130218 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rec.execute(self) Aug 29 22:29:55.130218 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 29 22:29:55.130218 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 29 22:29:55.136926 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 29 22:29:55.136926 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 29 22:29:55.136926 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 29 22:29:55.136926 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 29 22:29:55.136926 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 29 22:29:55.136926 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 29 22:29:55.136926 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 29 22:29:55.136926 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 29 22:29:55.136926 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 29 22:29:55.136926 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 29 22:29:55.136926 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 29 22:29:55.136926 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 29 22:29:55.136926 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 29 22:29:55.136926 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 29 22:29:55.136926 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 22:29:55.136926 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception Aug 29 22:29:55.138036 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 22:29:55.138036 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 22:29:55.138036 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 22:29:55.138036 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 22:29:55.138036 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 22:29:55.138036 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 22:29:55.138036 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 22:29:55.138036 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application conn.query(q) Aug 29 22:29:55.138036 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 22:29:55.138036 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 22:29:55.138036 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 22:29:55.138036 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application result.read() Aug 29 22:29:55.138036 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 22:29:55.138036 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 22:29:55.138036 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 22:29:55.138036 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 22:29:55.139091 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 22:29:55.139091 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 22:29:55.139091 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 22:29:55.139091 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 22:29:55.139091 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 29 22:29:55.139091 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 29 22:29:55.139091 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [parameters: {'user_id': 'c49b5f7a1dc24e77acae39db931e309d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 29 22:29:55.139091 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 29 22:29:55.139091 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:55.139091 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:29:55.139091 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:55.139091 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:55.139091 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:55.139091 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:55.139091 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:55.140114 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:55.140114 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:55.140114 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:55.140114 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:55.140114 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:55.140114 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:55.140114 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:55.140114 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 29 22:29:55.140114 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 29 22:29:55.140114 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:55.140114 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:55.140114 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:29:55.140114 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:55.140114 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:29:55.140114 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:55.140114 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 29 22:29:55.140114 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 29 22:29:55.141338 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 29 22:29:55.141338 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 29 22:29:55.141338 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 29 22:29:55.141338 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 29 22:29:55.141338 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 29 22:29:55.141338 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:55.141338 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 495/971] 10.222.0.52 () {60 vars in 1131 bytes} [Mon Aug 29 22:29:54 2022] POST /identity/v3/users => generated 160 bytes in 251 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:55.142790 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454e934-f8c7-40dc-8be8-d14c94548371 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:55.210806 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa75305-3f7a-4aa3-aadc-5958fa8b3ec8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:55.212748 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa75305-3f7a-4aa3-aadc-5958fa8b3ec8 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:55.215134 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:55.303520 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454e934-f8c7-40dc-8be8-d14c94548371 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:55.305385 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:55.351169 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30a26a4-46b1-4791-9591-5afb48e279e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:55.351615 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a26a4-46b1-4791-9591-5afb48e279e2 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:55.352004 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a26a4-46b1-4791-9591-5afb48e279e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:55.352394 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a26a4-46b1-4791-9591-5afb48e279e2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:55.360503 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30a26a4-46b1-4791-9591-5afb48e279e2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:55.361231 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 477/972] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:29:54 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 22:29:55.363092 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30a26a4-46b1-4791-9591-5afb48e279e2 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:55.365571 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa75305-3f7a-4aa3-aadc-5958fa8b3ec8 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:55.379681 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-a30a26a4-46b1-4791-9591-5afb48e279e2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 22:29:55.379681 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:55.379681 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:55.379681 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:55.379681 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:55.379681 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:55.379681 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:55.379681 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:55.379681 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:55.379681 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:55.379681 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:55.379681 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:55.379681 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:29:55.379681 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:29:55.379681 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:29:55.379681 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:29:55.381778 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:55.381778 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:55.381778 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:29:55.381778 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:55.381778 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:29:55.381778 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:29:55.381778 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:55.381778 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:55.381778 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:55.381778 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:55.381778 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:55.381778 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:55.381778 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:55.381778 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:55.381778 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:55.381778 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:55.381778 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:55.381778 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:55.383851 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:55.383851 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:55.383851 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:29:55.383851 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:29:55.383851 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:29:55.383851 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:29:55.383851 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:29:55.383851 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:29:55.383851 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 22:29:55.383851 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:29:55.385762 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 496/973] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:29:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:55.386904 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30a26a4-46b1-4791-9591-5afb48e279e2 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:55.530888 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30a26a4-46b1-4791-9591-5afb48e279e2 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:55.567940 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa75305-3f7a-4aa3-aadc-5958fa8b3ec8 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:55.570466 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:55.597029 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbabd111-71c6-47a7-a655-36f030da4489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:55.598154 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbabd111-71c6-47a7-a655-36f030da4489 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:55.598740 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbabd111-71c6-47a7-a655-36f030da4489 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:55.599221 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbabd111-71c6-47a7-a655-36f030da4489 None admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:55.600508 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbabd111-71c6-47a7-a655-36f030da4489 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:55.601663 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbabd111-71c6-47a7-a655-36f030da4489 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:55.603818 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbabd111-71c6-47a7-a655-36f030da4489 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:55.628043 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ba883b-b977-4909-b201-4341fd76f233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:55.629101 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba883b-b977-4909-b201-4341fd76f233 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:55.629556 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba883b-b977-4909-b201-4341fd76f233 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:55.630008 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba883b-b977-4909-b201-4341fd76f233 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:55.647656 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ba883b-b977-4909-b201-4341fd76f233 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:55.651094 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ba883b-b977-4909-b201-4341fd76f233 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:55.676149 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-95ba883b-b977-4909-b201-4341fd76f233 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:55.676149 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:55.676149 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:55.676149 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:55.676149 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:55.676149 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:55.676149 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:55.676149 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:55.676149 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:55.676149 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:55.676149 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:55.676149 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:55.676149 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:29:55.676149 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:29:55.676149 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:29:55.676149 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:29:55.678356 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:55.678356 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:55.678356 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:55.678356 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:55.678356 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:55.678356 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:55.678356 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:55.678356 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:55.678356 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:55.678356 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:55.678356 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:55.678356 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:55.678356 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:55.678356 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:55.678356 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:55.678356 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:55.678356 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:29:55.680143 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:29:55.680143 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:29:55.680143 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:55.685386 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 478/974] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:55.687839 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569dd458-49fb-4a4f-9b49-b80ee9a6a755 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:55.688154 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569dd458-49fb-4a4f-9b49-b80ee9a6a755 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:55.688394 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569dd458-49fb-4a4f-9b49-b80ee9a6a755 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:55.689047 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 497/975] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Aug 29 22:29:55 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:55.690634 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 479/976] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:29:55.691022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbabd111-71c6-47a7-a655-36f030da4489 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:55.703319 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af47f297-c3ba-48d5-8489-fad2c184ed05 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:55.703741 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af47f297-c3ba-48d5-8489-fad2c184ed05 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:55.703915 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af47f297-c3ba-48d5-8489-fad2c184ed05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:55.729823 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-af47f297-c3ba-48d5-8489-fad2c184ed05 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:55.730551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:55.731783 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-af47f297-c3ba-48d5-8489-fad2c184ed05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:55.819509 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbabd111-71c6-47a7-a655-36f030da4489 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:55.848930 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 480/977] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:29:55.870661 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3602cf-95c3-4dc4-a128-9786f032d45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:55.871441 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3602cf-95c3-4dc4-a128-9786f032d45c None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:55.871818 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3602cf-95c3-4dc4-a128-9786f032d45c None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:55.872192 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3602cf-95c3-4dc4-a128-9786f032d45c None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:55.873019 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3602cf-95c3-4dc4-a128-9786f032d45c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:55.873439 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3602cf-95c3-4dc4-a128-9786f032d45c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:55.874919 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3602cf-95c3-4dc4-a128-9786f032d45c None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:55.892304 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 498/978] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:29:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:55.898992 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af47f297-c3ba-48d5-8489-fad2c184ed05 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:56.010863 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:56.028487 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af47f297-c3ba-48d5-8489-fad2c184ed05 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:56.068908 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69722a5a-c7ed-4df8-b996-39a608240b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:56.069617 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69722a5a-c7ed-4df8-b996-39a608240b4d None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:56.070285 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69722a5a-c7ed-4df8-b996-39a608240b4d None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:56.070996 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69722a5a-c7ed-4df8-b996-39a608240b4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:56.079140 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69722a5a-c7ed-4df8-b996-39a608240b4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:56.080871 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69722a5a-c7ed-4df8-b996-39a608240b4d None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:56.093153 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-69722a5a-c7ed-4df8-b996-39a608240b4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:56.093153 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:56.093153 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:56.093153 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:56.093153 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:56.093153 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:56.093153 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:56.093153 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:56.093153 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:56.093153 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:56.093153 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:56.093153 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:56.093153 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:29:56.093153 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:29:56.093153 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:29:56.094110 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:29:56.094110 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:56.094110 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:56.094110 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:56.094110 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:56.094110 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:56.094110 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:56.094110 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:56.094110 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:56.094110 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:56.094110 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:56.094110 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:56.094110 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:56.094110 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:56.094110 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:56.094110 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:56.094110 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:56.102151 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:29:56.102151 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:29:56.102151 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:29:56.102151 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:29:56.102151 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:29:56.102151 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:29:56.102151 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:29:56.102151 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:56.102151 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 481/979] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:29:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:56.102665 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3602cf-95c3-4dc4-a128-9786f032d45c None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:56.124711 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69722a5a-c7ed-4df8-b996-39a608240b4d None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:56.198566 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:56.222240 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3602cf-95c3-4dc4-a128-9786f032d45c None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:56.246063 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69722a5a-c7ed-4df8-b996-39a608240b4d None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:56.262342 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e88a19-00a4-4c7f-835f-6a05970c43a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:56.262994 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e88a19-00a4-4c7f-835f-6a05970c43a9 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:56.263278 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e88a19-00a4-4c7f-835f-6a05970c43a9 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:56.263547 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e88a19-00a4-4c7f-835f-6a05970c43a9 None admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:56.264250 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e88a19-00a4-4c7f-835f-6a05970c43a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:56.264553 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e88a19-00a4-4c7f-835f-6a05970c43a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:56.266024 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e88a19-00a4-4c7f-835f-6a05970c43a9 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:56.281082 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 499/980] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Aug 29 22:29:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:29:56.287300 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e88a19-00a4-4c7f-835f-6a05970c43a9 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:56.292467 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffec93b7-df1b-40bd-9865-71ea31a5ccd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:56.293168 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec93b7-df1b-40bd-9865-71ea31a5ccd6 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:56.293446 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec93b7-df1b-40bd-9865-71ea31a5ccd6 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:56.293715 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec93b7-df1b-40bd-9865-71ea31a5ccd6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:56.294312 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffec93b7-df1b-40bd-9865-71ea31a5ccd6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:56.295722 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffec93b7-df1b-40bd-9865-71ea31a5ccd6 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:56.307363 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-ffec93b7-df1b-40bd-9865-71ea31a5ccd6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 22:29:56.307363 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:56.307363 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:29:56.307363 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:29:56.307363 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:29:56.307363 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:29:56.307363 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:29:56.307363 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:29:56.307363 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 22:29:56.307363 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:56.307363 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:29:56.307363 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:56.307363 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:56.307363 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:56.307363 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:56.307363 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:56.307363 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:56.308528 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:56.308528 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:56.308528 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:56.308528 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:56.308528 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:56.308528 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:56.308528 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:29:56.308528 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:29:56.308528 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:29:56.308528 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:29:56.308528 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:56.308528 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:56.308528 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:56.308528 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:56.308528 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:56.308528 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:56.309489 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:56.309489 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:56.309489 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:56.309489 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:56.309489 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:56.309489 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:56.309489 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:56.309489 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:56.309489 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:56.309489 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:56.309489 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:29:56.309489 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:29:56.309489 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 22:29:56.309489 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:56.309489 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 482/981] 10.222.0.52 () {56 vars in 1126 bytes} [Mon Aug 29 22:29:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:29:56.315238 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffec93b7-df1b-40bd-9865-71ea31a5ccd6 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:56.328101 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:56.412496 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e88a19-00a4-4c7f-835f-6a05970c43a9 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:56.440700 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffec93b7-df1b-40bd-9865-71ea31a5ccd6 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:56.451672 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7015c13a-a577-42c3-9a9c-b8a0193db345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:56.452685 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015c13a-a577-42c3-9a9c-b8a0193db345 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:56.453150 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015c13a-a577-42c3-9a9c-b8a0193db345 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:56.453429 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015c13a-a577-42c3-9a9c-b8a0193db345 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:56.454164 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7015c13a-a577-42c3-9a9c-b8a0193db345 None admin] RBAC: Adding query filter params (role.id=6912999fb1a046b48a32d8fd29c1922d, scope.project.id=ee84c3340f2641ad963655a7a212ef96, user.id=f77dfe78b1a8405084c06d9acee96511) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:56.454468 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7015c13a-a577-42c3-9a9c-b8a0193db345 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:56.456065 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7015c13a-a577-42c3-9a9c-b8a0193db345 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:56.466840 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 500/982] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:29:56 2022] GET /identity/v3/role_assignments?user.id=f77dfe78b1a8405084c06d9acee96511&scope.project.id=ee84c3340f2641ad963655a7a212ef96&role.id=6912999fb1a046b48a32d8fd29c1922d => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:56.486868 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f9105b-1991-4d2d-a408-ea6a377f8d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:56.488285 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f9105b-1991-4d2d-a408-ea6a377f8d68 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:56.488625 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f9105b-1991-4d2d-a408-ea6a377f8d68 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:56.488908 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f9105b-1991-4d2d-a408-ea6a377f8d68 None admin] PATH_INFO: `/v3/services` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:56.489739 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f9105b-1991-4d2d-a408-ea6a377f8d68 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:56.490139 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f9105b-1991-4d2d-a408-ea6a377f8d68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:56.491579 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f9105b-1991-4d2d-a408-ea6a377f8d68 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:56.502240 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 483/983] 10.222.0.52 () {56 vars in 1122 bytes} [Mon Aug 29 22:29:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:29:56.504324 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7015c13a-a577-42c3-9a9c-b8a0193db345 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:56.509143 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:56.642346 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7015c13a-a577-42c3-9a9c-b8a0193db345 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:56.687487 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7f0c5-c0f3-4360-8fd7-706f49c74f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:56.688407 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7f0c5-c0f3-4360-8fd7-706f49c74f70 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:56.688864 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7f0c5-c0f3-4360-8fd7-706f49c74f70 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:56.689274 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7f0c5-c0f3-4360-8fd7-706f49c74f70 None admin] PATH_INFO: `/v3/services` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:56.690143 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe7f0c5-c0f3-4360-8fd7-706f49c74f70 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:56.690599 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe7f0c5-c0f3-4360-8fd7-706f49c74f70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:56.692162 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe7f0c5-c0f3-4360-8fd7-706f49c74f70 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:56.698335 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 501/984] 10.222.0.52 () {56 vars in 1122 bytes} [Mon Aug 29 22:29:56 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:56.704202 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f9105b-1991-4d2d-a408-ea6a377f8d68 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:56.792300 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:56.826704 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f9105b-1991-4d2d-a408-ea6a377f8d68 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:56.852138 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efae370-2aff-493e-8627-8f35e0a3bbfa None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:56.852458 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efae370-2aff-493e-8627-8f35e0a3bbfa None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:56.852688 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efae370-2aff-493e-8627-8f35e0a3bbfa None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:56.854190 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 502/985] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:29:56.859910 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739541fc-2fe1-4855-bd02-d4d29865a101 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:56.860232 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739541fc-2fe1-4855-bd02-d4d29865a101 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:56.860475 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739541fc-2fe1-4855-bd02-d4d29865a101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:56.867188 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e39a8-6b05-4586-aee5-eeaea0bacf48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:56.868111 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e39a8-6b05-4586-aee5-eeaea0bacf48 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:56.868577 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e39a8-6b05-4586-aee5-eeaea0bacf48 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:56.869082 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e39a8-6b05-4586-aee5-eeaea0bacf48 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:56.869988 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497e39a8-6b05-4586-aee5-eeaea0bacf48 None admin] RBAC: Adding query filter params (interface=admin, service_id=ab30c5a3b5084acbbca53743095e472a, region_id=RegionOne) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:56.870555 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497e39a8-6b05-4586-aee5-eeaea0bacf48 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:56.872125 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497e39a8-6b05-4586-aee5-eeaea0bacf48 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:56.882373 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-739541fc-2fe1-4855-bd02-d4d29865a101 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:56.882587 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 484/986] 10.222.0.52 () {56 vars in 1249 bytes} [Mon Aug 29 22:29:56 2022] GET /identity/v3/endpoints?service_id=ab30c5a3b5084acbbca53743095e472a&interface=admin®ion_id=RegionOne => generated 193 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:56.882673 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-739541fc-2fe1-4855-bd02-d4d29865a101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:56.884038 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e39a8-6b05-4586-aee5-eeaea0bacf48 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:56.965436 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:56.971602 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 503/987] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:29:57.011594 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e39a8-6b05-4586-aee5-eeaea0bacf48 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:57.052933 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fa004d-fe1b-47a8-90aa-72eac0ece984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:57.053555 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa004d-fe1b-47a8-90aa-72eac0ece984 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:57.053837 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa004d-fe1b-47a8-90aa-72eac0ece984 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:57.054114 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa004d-fe1b-47a8-90aa-72eac0ece984 None admin] PATH_INFO: `/v3/services` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:57.054737 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fa004d-fe1b-47a8-90aa-72eac0ece984 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:57.056122 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fa004d-fe1b-47a8-90aa-72eac0ece984 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:57.058828 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:57.065507 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 485/988] 10.222.0.52 () {56 vars in 1087 bytes} [Mon Aug 29 22:29:56 2022] GET /identity/v3/services => generated 1497 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:29:57.265377 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739541fc-2fe1-4855-bd02-d4d29865a101 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:57.383464 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739541fc-2fe1-4855-bd02-d4d29865a101 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:57.423858 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781bb06b-015e-43be-af64-593b51e32f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:57.425075 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781bb06b-015e-43be-af64-593b51e32f5d None admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:57.425357 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781bb06b-015e-43be-af64-593b51e32f5d None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:57.425890 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781bb06b-015e-43be-af64-593b51e32f5d None admin] PATH_INFO: `/v3/services` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:57.426555 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781bb06b-015e-43be-af64-593b51e32f5d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:57.427927 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781bb06b-015e-43be-af64-593b51e32f5d None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:57.597422 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 504/989] 10.222.0.52 () {60 vars in 1141 bytes} [Mon Aug 29 22:29:57 2022] POST /identity/v3/services => generated 242 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:29:58.470120 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e0baa3-f7f8-4001-be8b-f648950b676a None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:58.470676 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e0baa3-f7f8-4001-be8b-f648950b676a None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:58.470676 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e0baa3-f7f8-4001-be8b-f648950b676a None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:58.472898 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 486/990] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:29:58.478617 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441b702-629f-4acc-93a6-241e81948821 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:58.479130 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441b702-629f-4acc-93a6-241e81948821 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:58.479130 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441b702-629f-4acc-93a6-241e81948821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:58.498585 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-7441b702-629f-4acc-93a6-241e81948821 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:58.499792 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-7441b702-629f-4acc-93a6-241e81948821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:58.563756 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:58.588881 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 505/991] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:58 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:29:58.856138 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e0baa3-f7f8-4001-be8b-f648950b676a None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:58.969900 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e0baa3-f7f8-4001-be8b-f648950b676a None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:59.010101 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686f75f6-2390-4cc1-8de2-89a9ba73702c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:59.011240 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686f75f6-2390-4cc1-8de2-89a9ba73702c None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:59.011240 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686f75f6-2390-4cc1-8de2-89a9ba73702c None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:59.011240 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686f75f6-2390-4cc1-8de2-89a9ba73702c None admin] PATH_INFO: `/v3/services/image` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:59.011821 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686f75f6-2390-4cc1-8de2-89a9ba73702c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:59.013257 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686f75f6-2390-4cc1-8de2-89a9ba73702c None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:59.023354 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-686f75f6-2390-4cc1-8de2-89a9ba73702c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 22:29:59.023354 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:59.023354 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:29:59.023354 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:29:59.023354 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:29:59.023354 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:29:59.023354 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:29:59.023354 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:29:59.023354 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 22:29:59.023354 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:59.023354 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:29:59.023354 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:59.023354 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:59.023354 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:59.023354 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:59.023354 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:59.023354 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:59.024419 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:59.024419 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:59.024419 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:59.024419 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:59.024419 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:59.024419 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:59.024419 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:29:59.024419 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:29:59.024419 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:29:59.024419 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:29:59.024419 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:59.024419 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:59.024419 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:59.024419 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:59.024419 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:59.024419 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:59.025298 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:59.025298 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:59.025298 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:59.025298 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:59.025298 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:59.025298 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:59.025298 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:59.025298 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:59.025298 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:59.025298 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:59.025298 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:29:59.025298 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:29:59.025298 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 22:29:59.025298 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:59.028889 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 487/992] 10.222.0.52 () {56 vars in 1105 bytes} [Mon Aug 29 22:29:58 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:59.030627 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7441b702-629f-4acc-93a6-241e81948821 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:59.144039 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbcfa5-28dc-4f43-ad6a-c70023e0ca7d None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:59.144237 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbcfa5-28dc-4f43-ad6a-c70023e0ca7d None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:59.144503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbcfa5-28dc-4f43-ad6a-c70023e0ca7d None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:59.146866 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 488/993] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:29:59.151358 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d99b0f4-c9d5-4f85-8643-53a5fd1bc131 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:59.151649 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d99b0f4-c9d5-4f85-8643-53a5fd1bc131 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:59.151929 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d99b0f4-c9d5-4f85-8643-53a5fd1bc131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:59.154289 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7441b702-629f-4acc-93a6-241e81948821 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:59.171129 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-9d99b0f4-c9d5-4f85-8643-53a5fd1bc131 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:59.172345 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-9d99b0f4-c9d5-4f85-8643-53a5fd1bc131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:59.195282 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795cd61-cd6c-4f3b-818e-9ed14b42dc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:59.195906 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795cd61-cd6c-4f3b-818e-9ed14b42dc6d None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:59.196185 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795cd61-cd6c-4f3b-818e-9ed14b42dc6d None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:59.196577 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795cd61-cd6c-4f3b-818e-9ed14b42dc6d None admin] PATH_INFO: `/v3/services` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:59.197236 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1795cd61-cd6c-4f3b-818e-9ed14b42dc6d None admin] RBAC: Adding query filter params (name=image) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:59.197535 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1795cd61-cd6c-4f3b-818e-9ed14b42dc6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:59.198945 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1795cd61-cd6c-4f3b-818e-9ed14b42dc6d None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:59.204070 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 506/994] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:59 2022] GET /identity/v3/services?name=image => generated 122 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:59.210292 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795cd61-cd6c-4f3b-818e-9ed14b42dc6d None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:59.243548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:59.251790 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 489/995] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:59 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:29:59.269909 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:59.328313 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795cd61-cd6c-4f3b-818e-9ed14b42dc6d None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:59.368678 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e709b85a-1820-47a6-89e4-4e46dedfb94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:59.370191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e709b85a-1820-47a6-89e4-4e46dedfb94e None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:59.370191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e709b85a-1820-47a6-89e4-4e46dedfb94e None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:59.370191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e709b85a-1820-47a6-89e4-4e46dedfb94e None admin] PATH_INFO: `/v3/services` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:59.370740 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e709b85a-1820-47a6-89e4-4e46dedfb94e None admin] RBAC: Adding query filter params (type=image) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:59.371048 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e709b85a-1820-47a6-89e4-4e46dedfb94e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:59.372426 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e709b85a-1820-47a6-89e4-4e46dedfb94e None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:59.385010 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 507/996] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:29:59 2022] GET /identity/v3/services?type=image => generated 122 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:29:59.532009 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d99b0f4-c9d5-4f85-8643-53a5fd1bc131 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:59.610189 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b25dff-1af2-4cb5-ba6c-a459d92b36a5 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:59.610757 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b25dff-1af2-4cb5-ba6c-a459d92b36a5 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:59.610757 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b25dff-1af2-4cb5-ba6c-a459d92b36a5 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:59.612899 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 508/997] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:29:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:29:59.618062 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01163278-8ee9-4ffd-b9d4-f9890cd12720 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:59.618347 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01163278-8ee9-4ffd-b9d4-f9890cd12720 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:59.618615 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01163278-8ee9-4ffd-b9d4-f9890cd12720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:59.637534 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-01163278-8ee9-4ffd-b9d4-f9890cd12720 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:29:59.638741 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-01163278-8ee9-4ffd-b9d4-f9890cd12720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:29:59.656048 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d99b0f4-c9d5-4f85-8643-53a5fd1bc131 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:59.697428 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302b1b7a-439b-484d-8ad0-5397de01fd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:59.698191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b1b7a-439b-484d-8ad0-5397de01fd96 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:59.698474 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b1b7a-439b-484d-8ad0-5397de01fd96 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:59.698746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b1b7a-439b-484d-8ad0-5397de01fd96 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:59.699450 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302b1b7a-439b-484d-8ad0-5397de01fd96 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:59.700853 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302b1b7a-439b-484d-8ad0-5397de01fd96 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:59.711503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-302b1b7a-439b-484d-8ad0-5397de01fd96 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 22:29:59.711503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:59.711503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:29:59.711503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:29:59.711503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:29:59.711503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:29:59.711503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:29:59.711503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:29:59.711503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 22:29:59.711503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:59.711503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:29:59.711503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:59.711503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:29:59.711503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:29:59.711503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:29:59.711503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:29:59.711503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:29:59.716847 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:29:59.716847 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:29:59.716847 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:29:59.716847 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:29:59.716847 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:29:59.716847 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:29:59.716847 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:29:59.716847 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:29:59.716847 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:29:59.716847 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:29:59.716847 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:29:59.716847 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:29:59.716847 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:29:59.716847 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:29:59.716847 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:29:59.716847 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:29:59.718585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:29:59.718585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:29:59.718585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:29:59.718585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:29:59.718585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:29:59.718585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:29:59.718585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:29:59.718585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:29:59.718585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:29:59.718585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:29:59.718585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:29:59.718585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:29:59.718585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 22:29:59.718585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:29:59.718585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 490/998] 10.222.0.52 () {56 vars in 1126 bytes} [Mon Aug 29 22:29:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:29:59.721413 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302b1b7a-439b-484d-8ad0-5397de01fd96 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:59.724224 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 509/999] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:29:59 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:29:59.786403 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:59.831845 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302b1b7a-439b-484d-8ad0-5397de01fd96 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:29:59.872569 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e19f1a-5454-4e57-a698-f1f746512e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:29:59.873217 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e19f1a-5454-4e57-a698-f1f746512e10 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:29:59.873497 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e19f1a-5454-4e57-a698-f1f746512e10 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:29:59.873769 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e19f1a-5454-4e57-a698-f1f746512e10 None admin] PATH_INFO: `/v3/services` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:29:59.874440 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e19f1a-5454-4e57-a698-f1f746512e10 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:29:59.875010 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e19f1a-5454-4e57-a698-f1f746512e10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:29:59.878048 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e19f1a-5454-4e57-a698-f1f746512e10 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:29:59.888970 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 491/1000] 10.222.0.52 () {56 vars in 1122 bytes} [Mon Aug 29 22:29:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:29:59.890702 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01163278-8ee9-4ffd-b9d4-f9890cd12720 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:29:59.942733 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:29:59.994432 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e19f1a-5454-4e57-a698-f1f746512e10 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:00.008347 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01163278-8ee9-4ffd-b9d4-f9890cd12720 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:00.052540 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f519e5-0318-4555-bd80-1da23b107b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:00.053431 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f519e5-0318-4555-bd80-1da23b107b4f None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:00.053949 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f519e5-0318-4555-bd80-1da23b107b4f None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:00.054403 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f519e5-0318-4555-bd80-1da23b107b4f None admin] PATH_INFO: `/v3/services` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:00.055225 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f519e5-0318-4555-bd80-1da23b107b4f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:30:00.055689 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f519e5-0318-4555-bd80-1da23b107b4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:00.057297 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f519e5-0318-4555-bd80-1da23b107b4f None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:00.063645 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 510/1001] 10.222.0.52 () {56 vars in 1122 bytes} [Mon Aug 29 22:29:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:00.070445 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f519e5-0318-4555-bd80-1da23b107b4f None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:00.120049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e19f1a-5454-4e57-a698-f1f746512e10 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:00.145325 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:00.157179 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da97df2f-ad0a-4684-812d-e2e18629ed18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:00.157815 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97df2f-ad0a-4684-812d-e2e18629ed18 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:00.158092 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97df2f-ad0a-4684-812d-e2e18629ed18 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:00.158406 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97df2f-ad0a-4684-812d-e2e18629ed18 None admin] PATH_INFO: `/v3/services/compute` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:00.159025 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da97df2f-ad0a-4684-812d-e2e18629ed18 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:00.161072 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da97df2f-ad0a-4684-812d-e2e18629ed18 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:00.171523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-da97df2f-ad0a-4684-812d-e2e18629ed18 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 22:30:00.171523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:30:00.171523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:30:00.171523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:30:00.171523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:30:00.171523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:30:00.171523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:30:00.171523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:30:00.171523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 22:30:00.171523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:30:00.171523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:30:00.171523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:30:00.171523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:30:00.171523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:30:00.171523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:30:00.171523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:30:00.171523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:30:00.172809 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:30:00.172809 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:30:00.172809 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:30:00.172809 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:30:00.172809 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:30:00.172809 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:30:00.172809 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:30:00.172809 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:30:00.172809 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:30:00.172809 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:30:00.172809 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:30:00.172809 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:30:00.172809 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:30:00.172809 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:30:00.172809 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:30:00.172809 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:30:00.173891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:30:00.173891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:30:00.173891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:30:00.173891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:30:00.173891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:30:00.173891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:30:00.173891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:30:00.173891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:30:00.173891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:30:00.173891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:30:00.173891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:30:00.173891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:30:00.173891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 22:30:00.173891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:30:00.182272 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 492/1002] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:29:59 2022] GET /identity/v3/services/compute => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:30:00.183453 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da97df2f-ad0a-4684-812d-e2e18629ed18 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:00.213810 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f519e5-0318-4555-bd80-1da23b107b4f None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:00.223960 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:00.255242 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbf43e3-af4f-42f9-9a3e-84a6ab4c6e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:00.255888 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbf43e3-af4f-42f9-9a3e-84a6ab4c6e82 None admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:00.256148 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbf43e3-af4f-42f9-9a3e-84a6ab4c6e82 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:00.256422 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbf43e3-af4f-42f9-9a3e-84a6ab4c6e82 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:00.257143 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbf43e3-af4f-42f9-9a3e-84a6ab4c6e82 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:00.258512 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbf43e3-af4f-42f9-9a3e-84a6ab4c6e82 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:00.304805 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da97df2f-ad0a-4684-812d-e2e18629ed18 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:00.342200 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3e6c2b-6342-4971-9a60-bc904686c34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:00.342819 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e6c2b-6342-4971-9a60-bc904686c34e None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:00.343096 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e6c2b-6342-4971-9a60-bc904686c34e None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:00.343364 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e6c2b-6342-4971-9a60-bc904686c34e None admin] PATH_INFO: `/v3/services` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:00.343995 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3e6c2b-6342-4971-9a60-bc904686c34e None admin] RBAC: Adding query filter params (name=compute) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:30:00.344296 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3e6c2b-6342-4971-9a60-bc904686c34e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:00.345734 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3e6c2b-6342-4971-9a60-bc904686c34e None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:00.357059 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 493/1003] 10.222.0.52 () {56 vars in 1112 bytes} [Mon Aug 29 22:30:00 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:00.360477 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3e6c2b-6342-4971-9a60-bc904686c34e None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:00.368827 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 511/1004] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:30:00 2022] POST /identity/v3/endpoints => generated 325 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:30:00.431671 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:00.496478 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3e6c2b-6342-4971-9a60-bc904686c34e None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:00.547222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172eee5b-f109-4941-a3a6-00c92893e7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:00.547852 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172eee5b-f109-4941-a3a6-00c92893e7be None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:00.548112 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172eee5b-f109-4941-a3a6-00c92893e7be None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:00.548385 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172eee5b-f109-4941-a3a6-00c92893e7be None admin] PATH_INFO: `/v3/services` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:00.549121 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172eee5b-f109-4941-a3a6-00c92893e7be None admin] RBAC: Adding query filter params (type=compute) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:30:00.549425 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172eee5b-f109-4941-a3a6-00c92893e7be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:00.551131 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172eee5b-f109-4941-a3a6-00c92893e7be None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:00.557657 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 494/1005] 10.222.0.52 () {56 vars in 1112 bytes} [Mon Aug 29 22:30:00 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:30:00.563569 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbf43e3-af4f-42f9-9a3e-84a6ab4c6e82 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:00.684436 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbf43e3-af4f-42f9-9a3e-84a6ab4c6e82 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:00.724446 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109caad4-f39c-4925-b64f-f8aa0b760fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:00.725105 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109caad4-f39c-4925-b64f-f8aa0b760fc6 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:00.725388 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109caad4-f39c-4925-b64f-f8aa0b760fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:00.725660 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109caad4-f39c-4925-b64f-f8aa0b760fc6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:00.726375 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109caad4-f39c-4925-b64f-f8aa0b760fc6 None admin] RBAC: Adding query filter params (interface=public, service_id=561f78eb18024af7865709abd0263237, region_id=RegionOne) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:30:00.726686 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109caad4-f39c-4925-b64f-f8aa0b760fc6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:00.728061 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109caad4-f39c-4925-b64f-f8aa0b760fc6 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:00.739216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 512/1006] 10.222.0.52 () {56 vars in 1251 bytes} [Mon Aug 29 22:30:00 2022] GET /identity/v3/endpoints?service_id=561f78eb18024af7865709abd0263237&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:00.740451 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172eee5b-f109-4941-a3a6-00c92893e7be None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:00.851797 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172eee5b-f109-4941-a3a6-00c92893e7be None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:00.863924 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:00.892091 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c4af42-798a-45ef-b4e0-1ec60bd19eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:00.892585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c4af42-798a-45ef-b4e0-1ec60bd19eca None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:00.892910 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c4af42-798a-45ef-b4e0-1ec60bd19eca None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:00.893558 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c4af42-798a-45ef-b4e0-1ec60bd19eca None admin] PATH_INFO: `/v3/services` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:00.894525 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c4af42-798a-45ef-b4e0-1ec60bd19eca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:00.896382 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c4af42-798a-45ef-b4e0-1ec60bd19eca None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:00.913185 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 495/1007] 10.222.0.52 () {56 vars in 1087 bytes} [Mon Aug 29 22:30:00 2022] GET /identity/v3/services => generated 1727 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:30:01.492990 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd9f94-1af5-460f-a9b5-03d763694a6c None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:01.501609 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd9f94-1af5-460f-a9b5-03d763694a6c None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:01.501609 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd9f94-1af5-460f-a9b5-03d763694a6c None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:01.501609 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 513/1008] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:30:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:30:01.501859 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adab5b8-f31f-4ef9-8673-60cd2f7a20e3 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:01.501859 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adab5b8-f31f-4ef9-8673-60cd2f7a20e3 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:01.501859 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adab5b8-f31f-4ef9-8673-60cd2f7a20e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:01.522015 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-1adab5b8-f31f-4ef9-8673-60cd2f7a20e3 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:30:01.523250 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-1adab5b8-f31f-4ef9-8673-60cd2f7a20e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:30:01.640785 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 496/1009] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:30:01 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:30:01.912030 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd9f94-1af5-460f-a9b5-03d763694a6c None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:01.957025 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:02.030651 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd9f94-1af5-460f-a9b5-03d763694a6c None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:02.069171 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58e5c0a-66a6-4fcc-97c9-3c87fed0929b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:02.069845 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e5c0a-66a6-4fcc-97c9-3c87fed0929b None admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:02.070130 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e5c0a-66a6-4fcc-97c9-3c87fed0929b None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:02.070402 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e5c0a-66a6-4fcc-97c9-3c87fed0929b None admin] PATH_INFO: `/v3/services` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:02.071052 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58e5c0a-66a6-4fcc-97c9-3c87fed0929b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:02.072407 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58e5c0a-66a6-4fcc-97c9-3c87fed0929b None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:02.284785 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 514/1010] 10.222.0.52 () {60 vars in 1141 bytes} [Mon Aug 29 22:30:01 2022] POST /identity/v3/services => generated 242 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:30:02.343778 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ac1dd-5018-48cf-b17d-095b403c3c20 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:02.344118 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ac1dd-5018-48cf-b17d-095b403c3c20 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:02.344320 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ac1dd-5018-48cf-b17d-095b403c3c20 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:02.346100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 497/1011] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:30:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:30:02.352881 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194fac90-f9d8-4292-a50f-bed850d58f36 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:02.353190 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194fac90-f9d8-4292-a50f-bed850d58f36 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:02.353442 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194fac90-f9d8-4292-a50f-bed850d58f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:02.373079 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-194fac90-f9d8-4292-a50f-bed850d58f36 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:30:02.374283 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-194fac90-f9d8-4292-a50f-bed850d58f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:30:02.464339 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 515/1012] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:30:02 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:30:02.779274 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ac1dd-5018-48cf-b17d-095b403c3c20 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:02.903650 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ac1dd-5018-48cf-b17d-095b403c3c20 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:02.906051 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:02.917776 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9db4f7-f917-4176-b90f-3fd66f376418 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:02.919784 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9db4f7-f917-4176-b90f-3fd66f376418 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:02.919784 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9db4f7-f917-4176-b90f-3fd66f376418 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:02.920900 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 516/1013] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:30:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:30:02.927302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8dd9c-c4a8-4b58-8e08-4f7ffc827245 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:02.927609 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8dd9c-c4a8-4b58-8e08-4f7ffc827245 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:02.927855 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8dd9c-c4a8-4b58-8e08-4f7ffc827245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:02.943689 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb3b942-0307-4451-bbf1-c8e451da8ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:02.944776 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb3b942-0307-4451-bbf1-c8e451da8ec8 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:02.945267 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb3b942-0307-4451-bbf1-c8e451da8ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:02.945748 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb3b942-0307-4451-bbf1-c8e451da8ec8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:02.947176 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb3b942-0307-4451-bbf1-c8e451da8ec8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:02.948653 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb3b942-0307-4451-bbf1-c8e451da8ec8 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:02.949301 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-87d8dd9c-c4a8-4b58-8e08-4f7ffc827245 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:30:02.950461 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 498/1014] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:30:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:02.950573 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-87d8dd9c-c4a8-4b58-8e08-4f7ffc827245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:30:02.956615 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb3b942-0307-4451-bbf1-c8e451da8ec8 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:02.966352 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:03.036887 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 517/1015] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:30:02 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:30:03.091522 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb3b942-0307-4451-bbf1-c8e451da8ec8 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:03.130342 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159f5557-7320-40a6-98c0-5103619bba7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:03.130950 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f5557-7320-40a6-98c0-5103619bba7c None admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:03.131231 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f5557-7320-40a6-98c0-5103619bba7c None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:03.131504 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f5557-7320-40a6-98c0-5103619bba7c None admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:03.132247 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159f5557-7320-40a6-98c0-5103619bba7c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:03.133708 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159f5557-7320-40a6-98c0-5103619bba7c None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:03.260769 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 499/1016] 10.222.0.52 () {60 vars in 1141 bytes} [Mon Aug 29 22:30:02 2022] POST /identity/v3/projects => generated 311 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:30:03.319437 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d8dd9c-c4a8-4b58-8e08-4f7ffc827245 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:03.455880 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d8dd9c-c4a8-4b58-8e08-4f7ffc827245 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:03.494739 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49231752-5e50-4355-a22d-4407882b3190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:03.495398 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49231752-5e50-4355-a22d-4407882b3190 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:03.495687 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49231752-5e50-4355-a22d-4407882b3190 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:03.495937 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49231752-5e50-4355-a22d-4407882b3190 None admin] PATH_INFO: `/v3/services/compute` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:03.496531 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49231752-5e50-4355-a22d-4407882b3190 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:03.498010 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49231752-5e50-4355-a22d-4407882b3190 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:03.508317 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-49231752-5e50-4355-a22d-4407882b3190 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 22:30:03.508317 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:30:03.508317 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:30:03.508317 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:30:03.508317 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:30:03.508317 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:30:03.508317 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:30:03.508317 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:30:03.508317 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 22:30:03.508317 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:30:03.508317 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:30:03.508317 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:30:03.508317 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:30:03.508317 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:30:03.508317 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:30:03.508317 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:30:03.508317 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:30:03.509571 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:30:03.509571 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:30:03.509571 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:30:03.509571 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:30:03.509571 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:30:03.509571 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:30:03.509571 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:30:03.509571 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:30:03.509571 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:30:03.509571 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:30:03.509571 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:30:03.509571 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:30:03.509571 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:30:03.509571 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:30:03.509571 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:30:03.509571 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:30:03.510542 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:30:03.510542 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:30:03.510542 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:30:03.510542 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:30:03.510542 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:30:03.510542 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:30:03.510542 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:30:03.510542 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:30:03.510542 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:30:03.510542 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:30:03.510542 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:30:03.510542 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:30:03.510542 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 22:30:03.510542 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:30:03.510542 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 518/1017] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:30:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:30:03.517729 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159f5557-7320-40a6-98c0-5103619bba7c None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:03.560923 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:03.645534 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159f5557-7320-40a6-98c0-5103619bba7c None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:03.681214 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bacdf79-3392-4db2-a441-9923fcb15f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:03.681828 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bacdf79-3392-4db2-a441-9923fcb15f7a None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:03.682109 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bacdf79-3392-4db2-a441-9923fcb15f7a None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:03.682385 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bacdf79-3392-4db2-a441-9923fcb15f7a None admin] PATH_INFO: `/v3/services` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:03.683015 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bacdf79-3392-4db2-a441-9923fcb15f7a None admin] RBAC: Adding query filter params (name=compute) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:30:03.683317 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bacdf79-3392-4db2-a441-9923fcb15f7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:03.684691 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bacdf79-3392-4db2-a441-9923fcb15f7a None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:03.694659 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 500/1018] 10.222.0.52 () {56 vars in 1112 bytes} [Mon Aug 29 22:30:03 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:30:03.695730 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49231752-5e50-4355-a22d-4407882b3190 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:03.710231 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:03.802902 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49231752-5e50-4355-a22d-4407882b3190 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:03.839038 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38bc92f-f75b-4987-9b81-7f4272d8faea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:03.839705 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38bc92f-f75b-4987-9b81-7f4272d8faea None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:03.839990 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38bc92f-f75b-4987-9b81-7f4272d8faea None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:03.840262 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38bc92f-f75b-4987-9b81-7f4272d8faea None admin] PATH_INFO: `/v3/services` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:03.841042 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38bc92f-f75b-4987-9b81-7f4272d8faea None admin] RBAC: Adding query filter params (type=compute) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:30:03.841409 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38bc92f-f75b-4987-9b81-7f4272d8faea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:03.842935 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38bc92f-f75b-4987-9b81-7f4272d8faea None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:03.854068 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 519/1019] 10.222.0.52 () {56 vars in 1112 bytes} [Mon Aug 29 22:30:03 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:03.855771 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bacdf79-3392-4db2-a441-9923fcb15f7a None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:03.952637 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:03.965627 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bacdf79-3392-4db2-a441-9923fcb15f7a None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:04.000092 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c17562-5f7d-4a36-9bd7-eedf8c5dc2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:04.000833 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c17562-5f7d-4a36-9bd7-eedf8c5dc2bf None admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:04.001131 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c17562-5f7d-4a36-9bd7-eedf8c5dc2bf None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:04.001302 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c17562-5f7d-4a36-9bd7-eedf8c5dc2bf None admin] PATH_INFO: `/v3/endpoints` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:04.001956 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c17562-5f7d-4a36-9bd7-eedf8c5dc2bf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:04.003308 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c17562-5f7d-4a36-9bd7-eedf8c5dc2bf None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:04.157545 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 501/1020] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:30:03 2022] POST /identity/v3/endpoints => generated 334 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:30:04.269011 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b442b95-ff87-4f13-847b-32be41a2ee77 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:04.269684 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b442b95-ff87-4f13-847b-32be41a2ee77 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:04.269684 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b442b95-ff87-4f13-847b-32be41a2ee77 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:04.270909 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 520/1021] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:30:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:30:04.276708 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b938bb47-a457-4808-9976-66f9e9f57f11 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:04.277036 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b938bb47-a457-4808-9976-66f9e9f57f11 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:04.277306 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b938bb47-a457-4808-9976-66f9e9f57f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:04.296617 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-b938bb47-a457-4808-9976-66f9e9f57f11 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:30:04.297837 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-b938bb47-a457-4808-9976-66f9e9f57f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:30:04.376234 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 502/1022] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:30:04 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:30:04.652438 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b442b95-ff87-4f13-847b-32be41a2ee77 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:04.733006 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:04.760199 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b442b95-ff87-4f13-847b-32be41a2ee77 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:04.795197 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72af1ffb-c590-43f3-aca1-d30713a00162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:04.796107 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72af1ffb-c590-43f3-aca1-d30713a00162 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:04.796583 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72af1ffb-c590-43f3-aca1-d30713a00162 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:04.797046 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72af1ffb-c590-43f3-aca1-d30713a00162 None admin] PATH_INFO: `/v3/services/image` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:04.797792 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72af1ffb-c590-43f3-aca1-d30713a00162 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:04.799965 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72af1ffb-c590-43f3-aca1-d30713a00162 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:04.809836 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-72af1ffb-c590-43f3-aca1-d30713a00162 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 22:30:04.809836 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:30:04.809836 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:30:04.809836 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:30:04.809836 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:30:04.809836 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:30:04.809836 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:30:04.809836 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:30:04.809836 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 22:30:04.809836 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:30:04.809836 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:30:04.809836 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:30:04.809836 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:30:04.809836 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:30:04.809836 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:30:04.809836 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:30:04.809836 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:30:04.810848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:30:04.810848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:30:04.810848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:30:04.810848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:30:04.810848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:30:04.810848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:30:04.810848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:30:04.810848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:30:04.810848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:30:04.810848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:30:04.810848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:30:04.810848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:30:04.810848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:30:04.810848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:30:04.810848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:30:04.810848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:30:04.811768 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:30:04.811768 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:30:04.811768 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:30:04.811768 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:30:04.811768 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:30:04.811768 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:30:04.811768 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:30:04.811768 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:30:04.811768 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:30:04.811768 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:30:04.811768 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:30:04.811768 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:30:04.811768 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 22:30:04.811768 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:30:04.813810 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 521/1023] 10.222.0.52 () {56 vars in 1105 bytes} [Mon Aug 29 22:30:04 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:30:04.820142 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b938bb47-a457-4808-9976-66f9e9f57f11 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:04.891724 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:04.923484 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b938bb47-a457-4808-9976-66f9e9f57f11 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:04.958845 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280119c9-9547-42f2-a7da-563051b9d93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:04.959452 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280119c9-9547-42f2-a7da-563051b9d93e None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:04.975117 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280119c9-9547-42f2-a7da-563051b9d93e None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:04.975117 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280119c9-9547-42f2-a7da-563051b9d93e None admin] PATH_INFO: `/v3/services` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:04.975117 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280119c9-9547-42f2-a7da-563051b9d93e None admin] RBAC: Adding query filter params (name=image) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:30:04.975117 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280119c9-9547-42f2-a7da-563051b9d93e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:04.975117 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280119c9-9547-42f2-a7da-563051b9d93e None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:04.975117 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 503/1024] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:30:04 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:04.975575 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72af1ffb-c590-43f3-aca1-d30713a00162 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:05.082966 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72af1ffb-c590-43f3-aca1-d30713a00162 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:05.118669 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4642513-290b-4aec-b998-32a227550601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:05.119273 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4642513-290b-4aec-b998-32a227550601 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:05.119548 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4642513-290b-4aec-b998-32a227550601 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:05.119815 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4642513-290b-4aec-b998-32a227550601 None admin] PATH_INFO: `/v3/services` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:05.120457 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4642513-290b-4aec-b998-32a227550601 None admin] RBAC: Adding query filter params (type=image) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:30:05.120813 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4642513-290b-4aec-b998-32a227550601 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:05.122187 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4642513-290b-4aec-b998-32a227550601 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:05.128639 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 522/1025] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:30:04 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:05.133688 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280119c9-9547-42f2-a7da-563051b9d93e None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:05.238962 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280119c9-9547-42f2-a7da-563051b9d93e None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:05.274433 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa3fec8-5edc-438a-8ebb-9d8331a2d555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:05.275039 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa3fec8-5edc-438a-8ebb-9d8331a2d555 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:05.275319 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa3fec8-5edc-438a-8ebb-9d8331a2d555 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:05.275591 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa3fec8-5edc-438a-8ebb-9d8331a2d555 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:05.276290 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa3fec8-5edc-438a-8ebb-9d8331a2d555 None admin] RBAC: Adding query filter params (interface=public, service_id=15bcb08ff3754e89b7d23a386b138306, region_id=RegionOne) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:30:05.276594 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa3fec8-5edc-438a-8ebb-9d8331a2d555 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:05.278099 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa3fec8-5edc-438a-8ebb-9d8331a2d555 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:05.279606 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:05.284990 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 504/1026] 10.222.0.52 () {56 vars in 1251 bytes} [Mon Aug 29 22:30:05 2022] GET /identity/v3/endpoints?service_id=15bcb08ff3754e89b7d23a386b138306&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:30:05.290658 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4642513-290b-4aec-b998-32a227550601 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:05.295554 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:05.307509 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdcbc66-c4f7-41ee-8a88-2029a1628dbf None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:05.307793 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdcbc66-c4f7-41ee-8a88-2029a1628dbf None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:05.308067 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdcbc66-c4f7-41ee-8a88-2029a1628dbf None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:05.309506 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 505/1027] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:30:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:30:05.315201 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b1f7a-6322-4245-9bae-5d035af751a2 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:05.315498 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b1f7a-6322-4245-9bae-5d035af751a2 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:05.315744 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b1f7a-6322-4245-9bae-5d035af751a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:05.334623 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-821b1f7a-6322-4245-9bae-5d035af751a2 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:30:05.335745 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-821b1f7a-6322-4245-9bae-5d035af751a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:30:05.412226 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4642513-290b-4aec-b998-32a227550601 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:05.420795 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 506/1028] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:30:05 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:30:05.456704 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab8d627-e2ed-4175-a8e7-b6fe9fc0871f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:05.457431 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8d627-e2ed-4175-a8e7-b6fe9fc0871f None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:05.457781 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8d627-e2ed-4175-a8e7-b6fe9fc0871f None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:05.458122 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8d627-e2ed-4175-a8e7-b6fe9fc0871f None admin] PATH_INFO: `/v3/services` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:05.458820 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab8d627-e2ed-4175-a8e7-b6fe9fc0871f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:05.460247 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab8d627-e2ed-4175-a8e7-b6fe9fc0871f None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:05.467350 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 523/1029] 10.222.0.52 () {56 vars in 1087 bytes} [Mon Aug 29 22:30:05 2022] GET /identity/v3/services => generated 1957 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:30:05.703557 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821b1f7a-6322-4245-9bae-5d035af751a2 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:05.818138 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821b1f7a-6322-4245-9bae-5d035af751a2 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:05.847378 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:05.855839 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0202611-9318-4a13-9b54-cb591ef42ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:05.856475 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0202611-9318-4a13-9b54-cb591ef42ccc None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:05.856780 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0202611-9318-4a13-9b54-cb591ef42ccc None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:05.861765 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0202611-9318-4a13-9b54-cb591ef42ccc None admin] PATH_INFO: `/v3/domains/default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:05.861765 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0202611-9318-4a13-9b54-cb591ef42ccc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:05.861765 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0202611-9318-4a13-9b54-cb591ef42ccc None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:05.861765 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 507/1030] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:30:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:05.868264 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab8d627-e2ed-4175-a8e7-b6fe9fc0871f None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:05.946400 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:05.983525 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab8d627-e2ed-4175-a8e7-b6fe9fc0871f None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:06.025347 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb29936-76ac-4cf7-950d-868e12d141d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:06.025954 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb29936-76ac-4cf7-950d-868e12d141d5 None admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:06.026235 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb29936-76ac-4cf7-950d-868e12d141d5 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:06.026507 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb29936-76ac-4cf7-950d-868e12d141d5 None admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:06.027162 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb29936-76ac-4cf7-950d-868e12d141d5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:06.028584 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb29936-76ac-4cf7-950d-868e12d141d5 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:06.231515 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 524/1031] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Aug 29 22:30:05 2022] POST /identity/v3/users => generated 285 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:30:06.243690 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d147b6-7716-4347-816a-15ed914faa22 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:06.244222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d147b6-7716-4347-816a-15ed914faa22 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:06.244222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d147b6-7716-4347-816a-15ed914faa22 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:06.245782 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 508/1032] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:30:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:30:06.251677 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a00b21-1718-4603-b8a7-fb27e41c6027 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:06.252779 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a00b21-1718-4603-b8a7-fb27e41c6027 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:06.253118 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a00b21-1718-4603-b8a7-fb27e41c6027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:06.273254 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-49a00b21-1718-4603-b8a7-fb27e41c6027 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:30:06.274962 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-49a00b21-1718-4603-b8a7-fb27e41c6027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:30:06.354140 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 525/1033] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:30:06 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:30:06.630368 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d147b6-7716-4347-816a-15ed914faa22 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:06.738899 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d147b6-7716-4347-816a-15ed914faa22 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:06.776559 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513e751a-e3f1-439b-ac03-5413bee1930d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:06.777221 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e751a-e3f1-439b-ac03-5413bee1930d None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:06.777504 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e751a-e3f1-439b-ac03-5413bee1930d None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:06.777776 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e751a-e3f1-439b-ac03-5413bee1930d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:06.789277 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513e751a-e3f1-439b-ac03-5413bee1930d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:30:06.794104 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-513e751a-e3f1-439b-ac03-5413bee1930d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 22:30:06.794104 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:30:06.794104 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:30:06.794104 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:30:06.794104 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:30:06.794104 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:30:06.794104 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:30:06.794104 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:30:06.794104 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:30:06.794104 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:30:06.794104 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:30:06.794104 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:30:06.794104 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:30:06.794104 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:30:06.794104 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:30:06.794104 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:30:06.795030 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:30:06.795030 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:30:06.795030 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:30:06.795030 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:30:06.795030 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:30:06.795030 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:30:06.795030 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:30:06.795030 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:30:06.795030 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:30:06.795030 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:30:06.795030 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:30:06.795030 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:30:06.795030 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:30:06.795030 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:30:06.795030 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:30:06.795030 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:30:06.795893 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:30:06.795893 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:30:06.795893 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:30:06.795893 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:30:06.795893 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:30:06.795893 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:30:06.795893 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:30:06.795893 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:30:06.795893 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:30:06.795893 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:30:06.795893 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 22:30:06.795893 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:30:06.800506 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 509/1034] 10.222.0.52 () {56 vars in 1120 bytes} [Mon Aug 29 22:30:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:30:06.802119 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a00b21-1718-4603-b8a7-fb27e41c6027 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:06.858459 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:06.907682 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a00b21-1718-4603-b8a7-fb27e41c6027 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:06.945915 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def8ffa-355c-411d-8644-824dbfce9a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:06.946541 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def8ffa-355c-411d-8644-824dbfce9a9e None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:06.946822 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def8ffa-355c-411d-8644-824dbfce9a9e None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:06.947150 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def8ffa-355c-411d-8644-824dbfce9a9e None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:06.947825 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2def8ffa-355c-411d-8644-824dbfce9a9e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:30:06.948127 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2def8ffa-355c-411d-8644-824dbfce9a9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:06.949655 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2def8ffa-355c-411d-8644-824dbfce9a9e None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:06.959881 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 526/1035] 10.222.0.52 () {56 vars in 1115 bytes} [Mon Aug 29 22:30:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:06.965676 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513e751a-e3f1-439b-ac03-5413bee1930d None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:07.028340 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:07.075354 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513e751a-e3f1-439b-ac03-5413bee1930d None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:07.110925 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8e25c-fc33-4b99-8f09-764dc2aff0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:07.111608 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8e25c-fc33-4b99-8f09-764dc2aff0e8 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:07.111887 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8e25c-fc33-4b99-8f09-764dc2aff0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:07.112337 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8e25c-fc33-4b99-8f09-764dc2aff0e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:07.123618 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef8e25c-fc33-4b99-8f09-764dc2aff0e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:07.125096 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef8e25c-fc33-4b99-8f09-764dc2aff0e8 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:07.140700 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-cef8e25c-fc33-4b99-8f09-764dc2aff0e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:30:07.140700 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:30:07.140700 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:30:07.140700 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:30:07.140700 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:30:07.140700 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:30:07.140700 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:30:07.140700 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:30:07.140700 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:30:07.140700 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:30:07.140700 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:30:07.140700 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:30:07.140700 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:30:07.140700 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:30:07.140700 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:30:07.140700 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:30:07.141731 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:30:07.141731 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:30:07.141731 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:30:07.141731 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:30:07.141731 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:30:07.141731 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:30:07.141731 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:30:07.141731 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:30:07.141731 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:30:07.141731 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:30:07.141731 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:30:07.141731 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:30:07.141731 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:30:07.141731 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:30:07.141731 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:30:07.141731 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:30:07.141731 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:30:07.143336 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:30:07.143336 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:30:07.143336 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:30:07.147329 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 510/1036] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:30:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:30:07.148314 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def8ffa-355c-411d-8644-824dbfce9a9e None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:07.254472 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def8ffa-355c-411d-8644-824dbfce9a9e None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:07.269617 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:07.290973 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c7076-9da2-409d-be60-345e2914f154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:07.291656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c7076-9da2-409d-be60-345e2914f154 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:07.291948 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c7076-9da2-409d-be60-345e2914f154 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:07.292547 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c7076-9da2-409d-be60-345e2914f154 None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:07.293424 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c7076-9da2-409d-be60-345e2914f154 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:30:07.293982 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c7076-9da2-409d-be60-345e2914f154 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:07.295361 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c7076-9da2-409d-be60-345e2914f154 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:07.310099 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 527/1037] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:30:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:07.317014 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8e25c-fc33-4b99-8f09-764dc2aff0e8 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:07.434951 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7976f1b7-29b1-447e-9510-1956f44fba23 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:07.435462 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7976f1b7-29b1-447e-9510-1956f44fba23 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:07.435462 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7976f1b7-29b1-447e-9510-1956f44fba23 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:07.436983 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 528/1038] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:30:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:30:07.445363 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c063f2-a597-4dd1-a969-ca83822f876e None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:07.445802 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c063f2-a597-4dd1-a969-ca83822f876e None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:07.446206 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c063f2-a597-4dd1-a969-ca83822f876e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:07.447662 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8e25c-fc33-4b99-8f09-764dc2aff0e8 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:07.461982 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:07.467867 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-82c063f2-a597-4dd1-a969-ca83822f876e None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:30:07.469093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-82c063f2-a597-4dd1-a969-ca83822f876e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:30:07.503283 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef3de46-724c-4537-b703-ffd288a61555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:07.504247 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef3de46-724c-4537-b703-ffd288a61555 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:07.504822 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef3de46-724c-4537-b703-ffd288a61555 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:07.505235 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef3de46-724c-4537-b703-ffd288a61555 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:07.513671 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef3de46-724c-4537-b703-ffd288a61555 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:07.515122 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef3de46-724c-4537-b703-ffd288a61555 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:07.528049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-cef3de46-724c-4537-b703-ffd288a61555 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 22:30:07.528049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:30:07.528049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:30:07.528049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:30:07.528049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:30:07.528049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:30:07.528049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:30:07.528049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:30:07.528049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:30:07.528049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:30:07.528049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:30:07.528049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:30:07.528049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:30:07.528049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:30:07.528049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:30:07.528049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:30:07.529109 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:30:07.529109 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:30:07.529109 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:30:07.529109 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:30:07.529109 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:30:07.529109 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:30:07.529109 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:30:07.529109 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:30:07.529109 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:30:07.529109 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:30:07.529109 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:30:07.529109 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:30:07.529109 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:30:07.529109 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:30:07.529109 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:30:07.529109 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:30:07.529109 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:30:07.529109 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:30:07.530641 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:30:07.530641 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:30:07.530641 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:30:07.530641 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:30:07.530641 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:30:07.530641 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:30:07.530641 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:30:07.530641 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:30:07.530641 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 22:30:07.530641 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:30:07.530641 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 511/1039] 10.222.0.52 () {56 vars in 1093 bytes} [Mon Aug 29 22:30:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:30:07.535502 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef3de46-724c-4537-b703-ffd288a61555 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:07.570602 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 529/1040] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:30:07 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:30:07.655274 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef3de46-724c-4537-b703-ffd288a61555 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:07.694128 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd3f6f-df1a-4bd4-b287-4deb562e543a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:07.694735 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd3f6f-df1a-4bd4-b287-4deb562e543a None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:07.695009 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd3f6f-df1a-4bd4-b287-4deb562e543a None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:07.695276 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd3f6f-df1a-4bd4-b287-4deb562e543a None admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:07.695996 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecd3f6f-df1a-4bd4-b287-4deb562e543a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:30:07.696295 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecd3f6f-df1a-4bd4-b287-4deb562e543a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:07.697741 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecd3f6f-df1a-4bd4-b287-4deb562e543a None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:07.721273 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 512/1041] 10.222.0.52 () {56 vars in 1133 bytes} [Mon Aug 29 22:30:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:07.722842 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c063f2-a597-4dd1-a969-ca83822f876e None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:07.831588 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c063f2-a597-4dd1-a969-ca83822f876e None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:07.834262 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:07.852323 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd3f6f-df1a-4bd4-b287-4deb562e543a None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:07.870755 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7081a7-53ae-4323-a1c8-0306e4bcdd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:07.871669 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7081a7-53ae-4323-a1c8-0306e4bcdd13 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:07.872199 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7081a7-53ae-4323-a1c8-0306e4bcdd13 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:07.872787 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7081a7-53ae-4323-a1c8-0306e4bcdd13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:07.881861 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:07.947103 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7081a7-53ae-4323-a1c8-0306e4bcdd13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:07.948474 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7081a7-53ae-4323-a1c8-0306e4bcdd13 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:07.959181 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd3f6f-df1a-4bd4-b287-4deb562e543a None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:08.000282 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1148497c-0655-4b7f-b13e-17a423a5993d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:08.001228 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1148497c-0655-4b7f-b13e-17a423a5993d None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:08.001512 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1148497c-0655-4b7f-b13e-17a423a5993d None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:08.001785 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1148497c-0655-4b7f-b13e-17a423a5993d None admin] PATH_INFO: `/v3/services/image` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:08.002809 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1148497c-0655-4b7f-b13e-17a423a5993d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:08.004495 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1148497c-0655-4b7f-b13e-17a423a5993d None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:08.016785 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-1148497c-0655-4b7f-b13e-17a423a5993d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 22:30:08.016785 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:30:08.016785 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:30:08.016785 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:30:08.016785 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:30:08.016785 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:30:08.016785 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:30:08.016785 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:30:08.016785 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 22:30:08.016785 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:30:08.016785 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:30:08.016785 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:30:08.016785 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:30:08.016785 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:30:08.016785 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:30:08.016785 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:30:08.016785 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:30:08.021044 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:30:08.021044 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:30:08.021044 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:30:08.021044 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:30:08.021044 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:30:08.021044 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:30:08.021044 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:30:08.021044 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:30:08.021044 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:30:08.021044 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:30:08.021044 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:30:08.021044 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:30:08.021044 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:30:08.021044 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:30:08.021044 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:30:08.021044 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:30:08.021942 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:30:08.021942 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:30:08.021942 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:30:08.021942 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:30:08.021942 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:30:08.021942 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:30:08.021942 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:30:08.021942 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:30:08.021942 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:30:08.021942 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:30:08.021942 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:30:08.021942 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:30:08.021942 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 22:30:08.021942 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:30:08.021942 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 513/1042] 10.222.0.52 () {56 vars in 1105 bytes} [Mon Aug 29 22:30:07 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:30:08.024120 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1148497c-0655-4b7f-b13e-17a423a5993d None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:08.060106 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 530/1043] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:30:07 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 22:30:08.066062 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7081a7-53ae-4323-a1c8-0306e4bcdd13 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:08.069834 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:08.129559 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:08.139992 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1148497c-0655-4b7f-b13e-17a423a5993d None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:08.175445 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7081a7-53ae-4323-a1c8-0306e4bcdd13 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:08.177356 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d263fa4e-766b-4658-bb6c-f05f9cb4e9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:08.177968 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d263fa4e-766b-4658-bb6c-f05f9cb4e9de None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:08.178252 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d263fa4e-766b-4658-bb6c-f05f9cb4e9de None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:08.178525 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d263fa4e-766b-4658-bb6c-f05f9cb4e9de None admin] PATH_INFO: `/v3/services` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:08.179161 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d263fa4e-766b-4658-bb6c-f05f9cb4e9de None admin] RBAC: Adding query filter params (name=image) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:30:08.179528 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d263fa4e-766b-4658-bb6c-f05f9cb4e9de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:08.180937 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d263fa4e-766b-4658-bb6c-f05f9cb4e9de None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:08.191014 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 514/1044] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:30:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:08.192239 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d263fa4e-766b-4658-bb6c-f05f9cb4e9de None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:08.212077 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8283f-86f9-4100-ab5f-5f62cb4d1c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:08.212624 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8283f-86f9-4100-ab5f-5f62cb4d1c20 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:08.212958 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8283f-86f9-4100-ab5f-5f62cb4d1c20 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:08.213230 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8283f-86f9-4100-ab5f-5f62cb4d1c20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:08.224857 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d8283f-86f9-4100-ab5f-5f62cb4d1c20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:08.226294 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d8283f-86f9-4100-ab5f-5f62cb4d1c20 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:08.241246 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-e4d8283f-86f9-4100-ab5f-5f62cb4d1c20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:30:08.241246 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:30:08.241246 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:30:08.241246 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:30:08.241246 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:30:08.241246 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:30:08.241246 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:30:08.241246 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:30:08.241246 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:30:08.241246 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:30:08.241246 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:30:08.241246 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:30:08.241246 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:30:08.241246 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:30:08.241246 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:30:08.241246 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:30:08.242270 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:30:08.242270 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:30:08.242270 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:30:08.242270 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:30:08.242270 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:30:08.242270 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:30:08.242270 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:30:08.242270 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:30:08.242270 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:30:08.242270 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:30:08.242270 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:30:08.242270 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:30:08.242270 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:30:08.242270 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:30:08.242270 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:30:08.242270 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:30:08.242270 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:30:08.243181 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:30:08.243181 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:30:08.243181 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:30:08.243181 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 531/1045] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:30:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:30:08.247942 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8283f-86f9-4100-ab5f-5f62cb4d1c20 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:08.303502 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d263fa4e-766b-4658-bb6c-f05f9cb4e9de None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:08.307490 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:08.338540 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4adbef-b9a5-43cb-8097-05c8989f3072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:08.339141 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4adbef-b9a5-43cb-8097-05c8989f3072 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:08.339420 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4adbef-b9a5-43cb-8097-05c8989f3072 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:08.339697 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4adbef-b9a5-43cb-8097-05c8989f3072 None admin] PATH_INFO: `/v3/services` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:08.340320 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4adbef-b9a5-43cb-8097-05c8989f3072 None admin] RBAC: Adding query filter params (type=image) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:30:08.340627 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4adbef-b9a5-43cb-8097-05c8989f3072 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:08.342018 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4adbef-b9a5-43cb-8097-05c8989f3072 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:08.349142 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 515/1046] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:30:08 2022] GET /identity/v3/services?type=image => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:08.350398 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b85c8-ca8d-4fc4-bef0-e159029e2dea None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:08.350676 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b85c8-ca8d-4fc4-bef0-e159029e2dea None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:08.350946 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b85c8-ca8d-4fc4-bef0-e159029e2dea None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:08.352061 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8283f-86f9-4100-ab5f-5f62cb4d1c20 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:08.352755 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 516/1047] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:30:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:30:08.355363 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1b85c8-ca8d-4fc4-bef0-e159029e2dea None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:08.375712 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:08.390213 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e4c054-de24-471e-87c6-a2fc96966310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:08.390821 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e4c054-de24-471e-87c6-a2fc96966310 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:08.395069 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e4c054-de24-471e-87c6-a2fc96966310 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:08.395069 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e4c054-de24-471e-87c6-a2fc96966310 None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:08.395069 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e4c054-de24-471e-87c6-a2fc96966310 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:30:08.395069 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e4c054-de24-471e-87c6-a2fc96966310 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:08.395069 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e4c054-de24-471e-87c6-a2fc96966310 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:08.406835 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 532/1048] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:30:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:30:08.408120 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed82ea68-05f9-4a7a-9103-92b0d472d16c None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:08.409167 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed82ea68-05f9-4a7a-9103-92b0d472d16c None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:08.409503 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed82ea68-05f9-4a7a-9103-92b0d472d16c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:08.428078 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-ed82ea68-05f9-4a7a-9103-92b0d472d16c None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:30:08.429330 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-ed82ea68-05f9-4a7a-9103-92b0d472d16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:30:08.466301 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1b85c8-ca8d-4fc4-bef0-e159029e2dea None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:08.499768 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 533/1049] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:30:08 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:30:08.500706 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed82ea68-05f9-4a7a-9103-92b0d472d16c None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:08.510045 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e44a51f-7e0c-4fe1-b05e-5b9edeaadad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:08.510650 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e44a51f-7e0c-4fe1-b05e-5b9edeaadad5 None admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:08.510927 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e44a51f-7e0c-4fe1-b05e-5b9edeaadad5 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:08.511191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e44a51f-7e0c-4fe1-b05e-5b9edeaadad5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:08.511851 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e44a51f-7e0c-4fe1-b05e-5b9edeaadad5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:08.513270 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e44a51f-7e0c-4fe1-b05e-5b9edeaadad5 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:08.552859 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:08.605424 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed82ea68-05f9-4a7a-9103-92b0d472d16c None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:08.614969 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 517/1050] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:30:08 2022] POST /identity/v3/endpoints => generated 327 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:30:08.664611 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d3858c-e9a0-4476-807a-c60e67d2559c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:08.665303 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3858c-e9a0-4476-807a-c60e67d2559c None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:08.665584 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3858c-e9a0-4476-807a-c60e67d2559c None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:08.665854 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3858c-e9a0-4476-807a-c60e67d2559c None admin] PATH_INFO: `/v3/projects/service` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:08.674765 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d3858c-e9a0-4476-807a-c60e67d2559c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:08.676303 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d3858c-e9a0-4476-807a-c60e67d2559c None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:08.688848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-99d3858c-e9a0-4476-807a-c60e67d2559c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:30:08.688848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:30:08.688848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:30:08.688848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:30:08.688848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:30:08.688848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:30:08.688848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:30:08.688848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:30:08.688848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:30:08.688848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:30:08.688848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:30:08.688848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:30:08.688848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:30:08.688848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:30:08.688848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:30:08.695898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:30:08.695898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:30:08.695898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:30:08.695898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:30:08.695898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:30:08.695898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:30:08.695898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:30:08.695898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:30:08.695898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:30:08.695898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:30:08.695898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:30:08.695898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:30:08.695898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:30:08.695898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:30:08.695898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:30:08.695898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:30:08.695898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:30:08.696870 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:30:08.696870 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:30:08.696870 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:30:08.696870 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:30:08.696870 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:30:08.696870 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:30:08.696870 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:30:08.696870 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:30:08.696870 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 534/1051] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:30:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:30:08.697385 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e44a51f-7e0c-4fe1-b05e-5b9edeaadad5 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:08.777758 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d3858c-e9a0-4476-807a-c60e67d2559c None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:08.779279 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:08.808874 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e44a51f-7e0c-4fe1-b05e-5b9edeaadad5 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:08.849828 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5901a-b26a-4b90-a16e-50c08c151705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:08.850464 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5901a-b26a-4b90-a16e-50c08c151705 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:08.850741 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5901a-b26a-4b90-a16e-50c08c151705 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:08.851009 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5901a-b26a-4b90-a16e-50c08c151705 None admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:08.851738 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c5901a-b26a-4b90-a16e-50c08c151705 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:30:08.852050 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c5901a-b26a-4b90-a16e-50c08c151705 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:08.858063 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c5901a-b26a-4b90-a16e-50c08c151705 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:08.871963 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 518/1052] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Aug 29 22:30:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:08.878426 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5901a-b26a-4b90-a16e-50c08c151705 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:08.899387 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d3858c-e9a0-4476-807a-c60e67d2559c None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:08.901200 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:08.935990 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bf964d-0d9d-4d90-8110-11d34d276ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:08.936627 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf964d-0d9d-4d90-8110-11d34d276ee7 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:08.936928 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf964d-0d9d-4d90-8110-11d34d276ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:08.937343 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf964d-0d9d-4d90-8110-11d34d276ee7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:08.949003 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bf964d-0d9d-4d90-8110-11d34d276ee7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:30:08.954307 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-63bf964d-0d9d-4d90-8110-11d34d276ee7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:30:08.954307 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:30:08.954307 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:30:08.954307 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:30:08.954307 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:30:08.954307 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:30:08.954307 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:30:08.954307 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:30:08.954307 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:30:08.954307 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:30:08.954307 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:30:08.954307 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:30:08.954307 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:30:08.954307 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:30:08.954307 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:30:08.954307 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:30:08.961695 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:30:08.961695 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:30:08.961695 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:30:08.961695 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:30:08.961695 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:30:08.961695 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:30:08.961695 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:30:08.961695 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:30:08.961695 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:30:08.961695 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:30:08.961695 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:30:08.961695 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:30:08.961695 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:30:08.961695 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:30:08.961695 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:30:08.961695 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:30:08.962768 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:30:08.962768 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:30:08.962768 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:30:08.962768 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:30:08.962768 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:30:08.962768 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:30:08.962768 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:30:08.962768 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:30:08.962768 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:30:08.962768 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:30:08.962768 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:30:08.962768 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:30:08.962768 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 535/1053] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Aug 29 22:30:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:30:08.965421 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bf964d-0d9d-4d90-8110-11d34d276ee7 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:08.994910 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5901a-b26a-4b90-a16e-50c08c151705 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:09.038112 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e6991-cd7c-4762-81ed-c865c1c874b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:09.038615 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e6991-cd7c-4762-81ed-c865c1c874b3 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:09.038892 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e6991-cd7c-4762-81ed-c865c1c874b3 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:09.039160 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e6991-cd7c-4762-81ed-c865c1c874b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:09.039876 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9e6991-cd7c-4762-81ed-c865c1c874b3 None admin] RBAC: Adding query filter params (role.id=6912999fb1a046b48a32d8fd29c1922d, scope.project.id=ee84c3340f2641ad963655a7a212ef96, user.id=bdbf961e02934e88af3a72d6218f32fa) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:30:09.040185 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9e6991-cd7c-4762-81ed-c865c1c874b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:09.042031 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9e6991-cd7c-4762-81ed-c865c1c874b3 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:09.043860 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:09.049297 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 519/1054] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:30:08 2022] GET /identity/v3/role_assignments?user.id=bdbf961e02934e88af3a72d6218f32fa&scope.project.id=ee84c3340f2641ad963655a7a212ef96&role.id=6912999fb1a046b48a32d8fd29c1922d => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:30:09.093299 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bf964d-0d9d-4d90-8110-11d34d276ee7 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:09.136914 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab58bd-d530-4ec5-91ef-15d58ee96f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:09.137534 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab58bd-d530-4ec5-91ef-15d58ee96f29 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:09.137810 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab58bd-d530-4ec5-91ef-15d58ee96f29 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:09.138080 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab58bd-d530-4ec5-91ef-15d58ee96f29 None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:09.138753 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ab58bd-d530-4ec5-91ef-15d58ee96f29 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:30:09.139056 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ab58bd-d530-4ec5-91ef-15d58ee96f29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:09.140447 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ab58bd-d530-4ec5-91ef-15d58ee96f29 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:09.155978 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 536/1055] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:30:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:30:09.157125 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e6991-cd7c-4762-81ed-c865c1c874b3 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:09.270901 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e6991-cd7c-4762-81ed-c865c1c874b3 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:09.310489 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96bde82-c899-4e72-bff5-f44f93b495f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:09.311040 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96bde82-c899-4e72-bff5-f44f93b495f8 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:09.311319 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96bde82-c899-4e72-bff5-f44f93b495f8 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:09.311592 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96bde82-c899-4e72-bff5-f44f93b495f8 None admin] PATH_INFO: `/v3/users/90433f7da3c04bbfb8ffe2a3ec4d3491` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:09.325866 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96bde82-c899-4e72-bff5-f44f93b495f8 None admin] RBAC: Authorizing `identity:get_user(user_id=90433f7da3c04bbfb8ffe2a3ec4d3491)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:09.327312 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96bde82-c899-4e72-bff5-f44f93b495f8 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:09.332760 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 520/1056] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:30:09 2022] GET /identity/v3/users/90433f7da3c04bbfb8ffe2a3ec4d3491 => generated 285 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:09.335300 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab58bd-d530-4ec5-91ef-15d58ee96f29 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:09.362875 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:09.441936 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab58bd-d530-4ec5-91ef-15d58ee96f29 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:09.483554 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3795ad9-f808-488c-bd49-fec4f0dd18e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:09.484177 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3795ad9-f808-488c-bd49-fec4f0dd18e5 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:09.484447 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3795ad9-f808-488c-bd49-fec4f0dd18e5 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:09.484738 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3795ad9-f808-488c-bd49-fec4f0dd18e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:09.572614 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3795ad9-f808-488c-bd49-fec4f0dd18e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:09.574165 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3795ad9-f808-488c-bd49-fec4f0dd18e5 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:09.600206 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:09.701094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 537/1057] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:30:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 22:30:09.707888 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96bde82-c899-4e72-bff5-f44f93b495f8 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:09.773148 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:09.819498 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96bde82-c899-4e72-bff5-f44f93b495f8 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:09.855057 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ed07a-0105-474f-8615-c78311c6a4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:09.855601 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ed07a-0105-474f-8615-c78311c6a4b0 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:09.855880 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ed07a-0105-474f-8615-c78311c6a4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:09.856154 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ed07a-0105-474f-8615-c78311c6a4b0 None admin] PATH_INFO: `/v3/projects/72e07920945d4b0aa18d5ce7a1213a86` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:09.857478 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027ed07a-0105-474f-8615-c78311c6a4b0 None admin] RBAC: Authorizing `identity:get_project(project_id=72e07920945d4b0aa18d5ce7a1213a86)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:09.858738 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027ed07a-0105-474f-8615-c78311c6a4b0 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:09.860525 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 521/1058] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:30:09 2022] GET /identity/v3/projects/72e07920945d4b0aa18d5ce7a1213a86 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:09.866929 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3795ad9-f808-488c-bd49-fec4f0dd18e5 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:09.992835 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3795ad9-f808-488c-bd49-fec4f0dd18e5 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:10.031312 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:10.044765 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9edef5-a83f-480e-8583-e4d96e41fde2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:10.044765 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9edef5-a83f-480e-8583-e4d96e41fde2 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:10.044765 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9edef5-a83f-480e-8583-e4d96e41fde2 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:10.044765 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9edef5-a83f-480e-8583-e4d96e41fde2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:10.045436 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9edef5-a83f-480e-8583-e4d96e41fde2 None admin] RBAC: Adding query filter params (role.id=882f35bd847a406eb2222aeafc517a46, scope.project.id=72e07920945d4b0aa18d5ce7a1213a86, user.id=90433f7da3c04bbfb8ffe2a3ec4d3491) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:30:10.045834 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9edef5-a83f-480e-8583-e4d96e41fde2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:10.047264 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9edef5-a83f-480e-8583-e4d96e41fde2 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:10.054370 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 538/1059] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:30:09 2022] GET /identity/v3/role_assignments?user.id=90433f7da3c04bbfb8ffe2a3ec4d3491&scope.project.id=72e07920945d4b0aa18d5ce7a1213a86&role.id=882f35bd847a406eb2222aeafc517a46 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:10.653010 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc39e43-27a9-4b53-9069-59e892cd837e None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:10.653414 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc39e43-27a9-4b53-9069-59e892cd837e None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:10.653596 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc39e43-27a9-4b53-9069-59e892cd837e None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:10.654913 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 522/1060] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:30:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:30:10.661169 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278388e-7300-4f80-914c-3a406dba61d4 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:10.661642 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278388e-7300-4f80-914c-3a406dba61d4 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:10.661744 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278388e-7300-4f80-914c-3a406dba61d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:10.681075 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-0278388e-7300-4f80-914c-3a406dba61d4 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:30:10.682261 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-0278388e-7300-4f80-914c-3a406dba61d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:30:10.755034 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 539/1061] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:30:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:30:11.012075 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9218025e-7a0f-4e58-b7f6-1f1213251a57 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:11.012593 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9218025e-7a0f-4e58-b7f6-1f1213251a57 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:11.012593 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9218025e-7a0f-4e58-b7f6-1f1213251a57 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:11.014159 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 523/1062] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:30:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:30:11.019781 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aed47f-fb95-4142-b940-055861bb0f61 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:11.020072 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aed47f-fb95-4142-b940-055861bb0f61 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:11.020348 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aed47f-fb95-4142-b940-055861bb0f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:11.034254 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9218025e-7a0f-4e58-b7f6-1f1213251a57 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:11.039536 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-f3aed47f-fb95-4142-b940-055861bb0f61 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:30:11.040738 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-f3aed47f-fb95-4142-b940-055861bb0f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:30:11.116868 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 540/1063] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:30:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:30:11.158299 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9218025e-7a0f-4e58-b7f6-1f1213251a57 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:11.183287 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:11.195246 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb7440a-6b61-4878-a255-15b34670fdc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:11.195851 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7440a-6b61-4878-a255-15b34670fdc8 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:11.196182 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7440a-6b61-4878-a255-15b34670fdc8 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:11.196459 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7440a-6b61-4878-a255-15b34670fdc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:11.264844 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb7440a-6b61-4878-a255-15b34670fdc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:11.266135 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb7440a-6b61-4878-a255-15b34670fdc8 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:11.368754 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 524/1064] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:30:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 22:30:11.371430 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aed47f-fb95-4142-b940-055861bb0f61 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:11.390208 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb7440a-6b61-4878-a255-15b34670fdc8 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:11.417215 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:11.428023 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:11.481853 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aed47f-fb95-4142-b940-055861bb0f61 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:11.501069 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb7440a-6b61-4878-a255-15b34670fdc8 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:11.522972 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb09574e-bd8e-4525-9981-562812b59867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:11.523881 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09574e-bd8e-4525-9981-562812b59867 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:11.524353 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09574e-bd8e-4525-9981-562812b59867 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:11.524799 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09574e-bd8e-4525-9981-562812b59867 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:11.539204 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb09574e-bd8e-4525-9981-562812b59867 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:11.540401 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb09574e-bd8e-4525-9981-562812b59867 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:11.543461 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47523213-735b-4e9d-9b61-5e2155eceb81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:11.544113 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47523213-735b-4e9d-9b61-5e2155eceb81 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:11.544393 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47523213-735b-4e9d-9b61-5e2155eceb81 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:11.544665 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47523213-735b-4e9d-9b61-5e2155eceb81 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:11.557460 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47523213-735b-4e9d-9b61-5e2155eceb81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:30:11.558774 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-fb09574e-bd8e-4525-9981-562812b59867 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:30:11.558774 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:30:11.558774 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:30:11.558774 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:30:11.558774 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:30:11.558774 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:30:11.558774 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:30:11.558774 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:30:11.558774 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:30:11.558774 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:30:11.558774 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:30:11.558774 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:30:11.558774 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:30:11.558774 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:30:11.558774 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:30:11.558774 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:30:11.559687 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:30:11.559687 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:30:11.559687 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:30:11.559687 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:30:11.559687 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:30:11.559687 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:30:11.559687 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:30:11.559687 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:30:11.559687 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:30:11.559687 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:30:11.559687 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:30:11.559687 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:30:11.559687 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:30:11.559687 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:30:11.559687 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:30:11.559687 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:30:11.559687 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:30:11.566123 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:30:11.566123 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:30:11.566123 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:30:11.566123 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 541/1065] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:30:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:30:11.566352 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-47523213-735b-4e9d-9b61-5e2155eceb81 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 22:30:11.566352 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:30:11.566352 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:30:11.566352 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:30:11.566352 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:30:11.566352 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:30:11.566352 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:30:11.566352 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:30:11.566352 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:30:11.566352 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:30:11.566352 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:30:11.566352 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:30:11.566352 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:30:11.566352 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:30:11.566352 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:30:11.566352 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:30:11.567199 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:30:11.567199 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:30:11.567199 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:30:11.567199 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:30:11.567199 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:30:11.567199 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:30:11.567199 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:30:11.567199 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:30:11.567199 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:30:11.567199 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:30:11.567199 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:30:11.567199 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:30:11.567199 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:30:11.567199 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:30:11.567199 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:30:11.567199 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:30:11.569287 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:30:11.569287 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:30:11.569287 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:30:11.569287 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:30:11.569287 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:30:11.569287 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:30:11.569287 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:30:11.569287 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:30:11.569287 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:30:11.569287 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:30:11.569287 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 22:30:11.569287 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:30:11.569287 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 525/1066] 10.222.0.52 () {56 vars in 1120 bytes} [Mon Aug 29 22:30:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:30:11.571083 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb09574e-bd8e-4525-9981-562812b59867 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:11.571241 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47523213-735b-4e9d-9b61-5e2155eceb81 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:11.666014 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:11.682752 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:11.684746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47523213-735b-4e9d-9b61-5e2155eceb81 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:11.685981 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb09574e-bd8e-4525-9981-562812b59867 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:11.721934 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1011481-c40e-41d7-bd82-2f456da8bddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:11.721934 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1011481-c40e-41d7-bd82-2f456da8bddf None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:11.722262 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1011481-c40e-41d7-bd82-2f456da8bddf None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:11.722502 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1011481-c40e-41d7-bd82-2f456da8bddf None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:11.723142 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1011481-c40e-41d7-bd82-2f456da8bddf None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:30:11.723449 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1011481-c40e-41d7-bd82-2f456da8bddf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:11.724902 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1011481-c40e-41d7-bd82-2f456da8bddf None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:11.727943 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b02e98f-571e-45f2-b901-ea861495b5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:11.728547 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02e98f-571e-45f2-b901-ea861495b5f7 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:11.728868 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02e98f-571e-45f2-b901-ea861495b5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:11.729140 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02e98f-571e-45f2-b901-ea861495b5f7 None admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:11.729869 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b02e98f-571e-45f2-b901-ea861495b5f7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:30:11.730174 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b02e98f-571e-45f2-b901-ea861495b5f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:11.731533 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b02e98f-571e-45f2-b901-ea861495b5f7 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:11.739224 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 542/1067] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:30:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:11.745955 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 526/1068] 10.222.0.52 () {56 vars in 1115 bytes} [Mon Aug 29 22:30:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:30:11.747751 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1011481-c40e-41d7-bd82-2f456da8bddf None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:11.862725 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1011481-c40e-41d7-bd82-2f456da8bddf None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:11.902579 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd390c-cd59-430c-bafd-b4d0b83edb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:11.903216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd390c-cd59-430c-bafd-b4d0b83edb36 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:11.903498 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd390c-cd59-430c-bafd-b4d0b83edb36 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:11.903772 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd390c-cd59-430c-bafd-b4d0b83edb36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:11.915873 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfd390c-cd59-430c-bafd-b4d0b83edb36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:11.917379 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfd390c-cd59-430c-bafd-b4d0b83edb36 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:11.932980 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-1dfd390c-cd59-430c-bafd-b4d0b83edb36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:30:11.932980 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:30:11.932980 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:30:11.932980 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:30:11.932980 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:30:11.932980 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:30:11.932980 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:30:11.932980 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:30:11.932980 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:30:11.932980 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:30:11.932980 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:30:11.932980 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:30:11.932980 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:30:11.932980 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:30:11.932980 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:30:11.932980 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:30:11.933974 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:30:11.933974 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:30:11.933974 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:30:11.933974 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:30:11.933974 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:30:11.933974 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:30:11.933974 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:30:11.933974 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:30:11.933974 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:30:11.933974 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:30:11.933974 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:30:11.933974 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:30:11.933974 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:30:11.933974 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:30:11.933974 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:30:11.933974 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:30:11.933974 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:30:11.935241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:30:11.935241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:30:11.935241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:30:11.935241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 543/1069] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:30:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:30:11.939980 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b02e98f-571e-45f2-b901-ea861495b5f7 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:11.984213 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a356334-df93-47ae-8b1e-e68843b75625 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:11.984213 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a356334-df93-47ae-8b1e-e68843b75625 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:11.984213 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a356334-df93-47ae-8b1e-e68843b75625 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:11.985384 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 544/1070] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:30:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:30:11.994316 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb3d64-b7b8-48b1-aaee-a14da94fab3f None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:11.994916 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb3d64-b7b8-48b1-aaee-a14da94fab3f None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:11.995190 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb3d64-b7b8-48b1-aaee-a14da94fab3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:12.013757 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-a9cb3d64-b7b8-48b1-aaee-a14da94fab3f None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:30:12.015169 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-a9cb3d64-b7b8-48b1-aaee-a14da94fab3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:30:12.025704 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:12.058502 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b02e98f-571e-45f2-b901-ea861495b5f7 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:12.091813 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 545/1071] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:30:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:30:12.093265 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a00fa6-10dc-4db4-9807-f5fc36ac8c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:12.093862 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a00fa6-10dc-4db4-9807-f5fc36ac8c4c None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:12.094136 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a00fa6-10dc-4db4-9807-f5fc36ac8c4c None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:12.094405 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a00fa6-10dc-4db4-9807-f5fc36ac8c4c None admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:12.095034 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a00fa6-10dc-4db4-9807-f5fc36ac8c4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:30:12.095330 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a00fa6-10dc-4db4-9807-f5fc36ac8c4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:12.096739 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a00fa6-10dc-4db4-9807-f5fc36ac8c4c None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:12.108359 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 527/1072] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:30:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:12.115159 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb3d64-b7b8-48b1-aaee-a14da94fab3f None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:12.214543 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb3d64-b7b8-48b1-aaee-a14da94fab3f None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:12.249834 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358dddf-5da6-4d3d-b01d-8397c81df503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:12.250451 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358dddf-5da6-4d3d-b01d-8397c81df503 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:12.250725 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358dddf-5da6-4d3d-b01d-8397c81df503 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:12.250993 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358dddf-5da6-4d3d-b01d-8397c81df503 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:12.258759 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2358dddf-5da6-4d3d-b01d-8397c81df503 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:12.260183 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2358dddf-5da6-4d3d-b01d-8397c81df503 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:12.271437 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-2358dddf-5da6-4d3d-b01d-8397c81df503 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 22:30:12.271437 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:30:12.271437 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:30:12.271437 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:30:12.271437 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:30:12.271437 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:30:12.271437 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:30:12.271437 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:30:12.271437 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:30:12.271437 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:30:12.271437 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:30:12.271437 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:30:12.271437 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:30:12.271437 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:30:12.271437 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:30:12.271437 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:30:12.272629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:30:12.272629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:30:12.272629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:30:12.272629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:30:12.272629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:30:12.272629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:30:12.272629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:30:12.272629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:30:12.272629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:30:12.272629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:30:12.272629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:30:12.272629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:30:12.272629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:30:12.272629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:30:12.272629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:30:12.272629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:30:12.272629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:30:12.272629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:30:12.273658 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:30:12.273658 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:30:12.273658 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:30:12.273658 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:30:12.273658 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:30:12.273658 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:30:12.273658 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:30:12.273658 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:30:12.273658 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 22:30:12.273658 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:30:12.273658 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 546/1073] 10.222.0.52 () {56 vars in 1093 bytes} [Mon Aug 29 22:30:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:30:12.278369 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a00fa6-10dc-4db4-9807-f5fc36ac8c4c None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:12.287936 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:12.363302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358dddf-5da6-4d3d-b01d-8397c81df503 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:12.366290 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:12.386557 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a00fa6-10dc-4db4-9807-f5fc36ac8c4c None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:12.423682 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb853c6f-ae47-47e2-8553-0bc7aacd4934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:12.427517 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb853c6f-ae47-47e2-8553-0bc7aacd4934 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:12.427517 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb853c6f-ae47-47e2-8553-0bc7aacd4934 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:12.427517 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb853c6f-ae47-47e2-8553-0bc7aacd4934 None admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:12.427517 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb853c6f-ae47-47e2-8553-0bc7aacd4934 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:30:12.427517 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb853c6f-ae47-47e2-8553-0bc7aacd4934 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:12.429120 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb853c6f-ae47-47e2-8553-0bc7aacd4934 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:12.449590 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 528/1074] 10.222.0.52 () {56 vars in 1133 bytes} [Mon Aug 29 22:30:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:12.455270 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb853c6f-ae47-47e2-8553-0bc7aacd4934 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:12.486026 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358dddf-5da6-4d3d-b01d-8397c81df503 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:12.526471 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d594da5f-7801-42a3-9727-94c423168e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:12.527448 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594da5f-7801-42a3-9727-94c423168e88 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:12.527910 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594da5f-7801-42a3-9727-94c423168e88 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:12.528362 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594da5f-7801-42a3-9727-94c423168e88 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:12.529519 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:12.542774 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d594da5f-7801-42a3-9727-94c423168e88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:30:12.548137 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-d594da5f-7801-42a3-9727-94c423168e88 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:30:12.548137 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:30:12.548137 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:30:12.548137 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:30:12.548137 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:30:12.548137 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:30:12.548137 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:30:12.548137 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:30:12.548137 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:30:12.548137 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:30:12.548137 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:30:12.548137 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:30:12.548137 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:30:12.548137 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:30:12.548137 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:30:12.548137 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:30:12.550202 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:30:12.550202 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:30:12.550202 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:30:12.550202 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:30:12.550202 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:30:12.550202 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:30:12.550202 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:30:12.550202 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:30:12.550202 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:30:12.550202 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:30:12.550202 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:30:12.550202 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:30:12.550202 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:30:12.550202 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:30:12.550202 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:30:12.550202 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:30:12.551552 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:30:12.551552 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:30:12.551552 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:30:12.551552 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:30:12.551552 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:30:12.551552 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:30:12.551552 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:30:12.551552 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:30:12.551552 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:30:12.551552 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:30:12.551552 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:30:12.551552 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:30:12.551552 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 547/1075] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Aug 29 22:30:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:30:12.556232 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d594da5f-7801-42a3-9727-94c423168e88 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:12.578966 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb853c6f-ae47-47e2-8553-0bc7aacd4934 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:12.615110 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1643d80-0f1e-4b76-8be9-3a6678491b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:12.615738 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1643d80-0f1e-4b76-8be9-3a6678491b2b None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:12.616019 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1643d80-0f1e-4b76-8be9-3a6678491b2b None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:12.616346 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1643d80-0f1e-4b76-8be9-3a6678491b2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:12.620291 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:12.627059 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1643d80-0f1e-4b76-8be9-3a6678491b2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:12.628705 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1643d80-0f1e-4b76-8be9-3a6678491b2b None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:12.645204 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-f1643d80-0f1e-4b76-8be9-3a6678491b2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:30:12.645204 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:30:12.645204 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:30:12.645204 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:30:12.645204 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:30:12.645204 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:30:12.645204 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:30:12.645204 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:30:12.645204 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:30:12.645204 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:30:12.645204 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:30:12.645204 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:30:12.645204 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:30:12.645204 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:30:12.645204 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:30:12.645204 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:30:12.646803 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:30:12.646803 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:30:12.646803 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:30:12.646803 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:30:12.646803 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:30:12.646803 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:30:12.646803 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:30:12.646803 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:30:12.646803 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:30:12.646803 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:30:12.646803 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:30:12.646803 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:30:12.646803 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:30:12.646803 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:30:12.646803 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:30:12.646803 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:30:12.646803 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:30:12.647703 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:30:12.647703 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:30:12.647703 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:30:12.647703 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 529/1076] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:30:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:30:12.652309 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1643d80-0f1e-4b76-8be9-3a6678491b2b None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:12.671643 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d594da5f-7801-42a3-9727-94c423168e88 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:12.715261 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f69ed8-d5a7-463d-bb13-53f2639f5430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:12.716320 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f69ed8-d5a7-463d-bb13-53f2639f5430 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:12.716864 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f69ed8-d5a7-463d-bb13-53f2639f5430 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:12.717349 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f69ed8-d5a7-463d-bb13-53f2639f5430 None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:12.718238 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f69ed8-d5a7-463d-bb13-53f2639f5430 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:30:12.718546 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f69ed8-d5a7-463d-bb13-53f2639f5430 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:12.719916 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f69ed8-d5a7-463d-bb13-53f2639f5430 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:12.729327 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 548/1077] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:30:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:30:12.735189 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f69ed8-d5a7-463d-bb13-53f2639f5430 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:12.765118 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1643d80-0f1e-4b76-8be9-3a6678491b2b None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:12.782027 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:12.800397 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c71f65c-5c59-49bc-8bb9-eb3d1452d5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:12.805318 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71f65c-5c59-49bc-8bb9-eb3d1452d5ad None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:12.805318 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71f65c-5c59-49bc-8bb9-eb3d1452d5ad None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:12.805318 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71f65c-5c59-49bc-8bb9-eb3d1452d5ad None admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:12.805318 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c71f65c-5c59-49bc-8bb9-eb3d1452d5ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:30:12.805318 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c71f65c-5c59-49bc-8bb9-eb3d1452d5ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:12.805318 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c71f65c-5c59-49bc-8bb9-eb3d1452d5ad None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:12.832687 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 530/1078] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:30:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:30:12.834438 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c71f65c-5c59-49bc-8bb9-eb3d1452d5ad None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:12.869943 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f69ed8-d5a7-463d-bb13-53f2639f5430 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:12.898229 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:12.914198 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f672ac62-1b69-4a17-a49f-d897dc7d5b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:12.914868 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672ac62-1b69-4a17-a49f-d897dc7d5b73 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:12.915156 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672ac62-1b69-4a17-a49f-d897dc7d5b73 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:12.915444 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672ac62-1b69-4a17-a49f-d897dc7d5b73 None admin] PATH_INFO: `/v3/users/90433f7da3c04bbfb8ffe2a3ec4d3491` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:12.916915 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f672ac62-1b69-4a17-a49f-d897dc7d5b73 None admin] RBAC: Authorizing `identity:get_user(user_id=90433f7da3c04bbfb8ffe2a3ec4d3491)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:12.918374 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f672ac62-1b69-4a17-a49f-d897dc7d5b73 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:12.925848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 549/1079] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:30:12 2022] GET /identity/v3/users/90433f7da3c04bbfb8ffe2a3ec4d3491 => generated 285 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:12.927278 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f672ac62-1b69-4a17-a49f-d897dc7d5b73 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:12.963835 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c71f65c-5c59-49bc-8bb9-eb3d1452d5ad None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:13.001241 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84321dc1-fc0e-4afc-8887-43d18f62dbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:13.001917 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84321dc1-fc0e-4afc-8887-43d18f62dbf2 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:13.002585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84321dc1-fc0e-4afc-8887-43d18f62dbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:13.002585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84321dc1-fc0e-4afc-8887-43d18f62dbf2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:13.010226 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84321dc1-fc0e-4afc-8887-43d18f62dbf2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:13.011706 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84321dc1-fc0e-4afc-8887-43d18f62dbf2 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:13.024511 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-84321dc1-fc0e-4afc-8887-43d18f62dbf2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:30:13.024511 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:30:13.024511 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:30:13.024511 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:30:13.024511 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:30:13.024511 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:30:13.024511 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:30:13.024511 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:30:13.024511 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:30:13.024511 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:30:13.024511 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:30:13.024511 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:30:13.024511 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:30:13.024511 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:30:13.024511 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:30:13.025550 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:30:13.025550 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:30:13.025550 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:30:13.025550 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:30:13.025550 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:30:13.025550 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:30:13.025550 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:30:13.025550 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:30:13.025550 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:30:13.025550 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:30:13.025550 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:30:13.025550 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:30:13.025550 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:30:13.025550 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:30:13.025550 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:30:13.025550 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:30:13.025550 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:30:13.026794 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:30:13.026794 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:30:13.026794 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:30:13.026794 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:30:13.026794 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:30:13.026794 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:30:13.026794 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:30:13.026794 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:30:13.026794 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 531/1080] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:30:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:30:13.031806 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84321dc1-fc0e-4afc-8887-43d18f62dbf2 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:13.058706 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f672ac62-1b69-4a17-a49f-d897dc7d5b73 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:13.083590 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:13.121229 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be86c827-dd05-48fe-bbb1-a47ce6dd03a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:13.121464 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be86c827-dd05-48fe-bbb1-a47ce6dd03a3 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:13.121667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be86c827-dd05-48fe-bbb1-a47ce6dd03a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:13.121908 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be86c827-dd05-48fe-bbb1-a47ce6dd03a3 None admin] PATH_INFO: `/v3/projects/72e07920945d4b0aa18d5ce7a1213a86` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:13.124180 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be86c827-dd05-48fe-bbb1-a47ce6dd03a3 None admin] RBAC: Authorizing `identity:get_project(project_id=72e07920945d4b0aa18d5ce7a1213a86)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:13.126030 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be86c827-dd05-48fe-bbb1-a47ce6dd03a3 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:13.128802 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 550/1081] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:30:12 2022] GET /identity/v3/projects/72e07920945d4b0aa18d5ce7a1213a86 => generated 311 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:13.137677 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be86c827-dd05-48fe-bbb1-a47ce6dd03a3 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:13.175697 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84321dc1-fc0e-4afc-8887-43d18f62dbf2 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:13.222822 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:13.224289 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843beaa9-dfe9-419f-827e-b4c03825a027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:13.225069 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843beaa9-dfe9-419f-827e-b4c03825a027 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:13.225232 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843beaa9-dfe9-419f-827e-b4c03825a027 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:13.225509 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843beaa9-dfe9-419f-827e-b4c03825a027 None admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:13.226205 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843beaa9-dfe9-419f-827e-b4c03825a027 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:30:13.226511 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843beaa9-dfe9-419f-827e-b4c03825a027 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:13.227980 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843beaa9-dfe9-419f-827e-b4c03825a027 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:13.246436 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 532/1082] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Aug 29 22:30:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:30:13.253819 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843beaa9-dfe9-419f-827e-b4c03825a027 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:13.269582 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be86c827-dd05-48fe-bbb1-a47ce6dd03a3 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:13.312009 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747b49e8-5f20-43ce-978d-7d3253650c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:13.312677 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747b49e8-5f20-43ce-978d-7d3253650c86 None admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:13.313009 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747b49e8-5f20-43ce-978d-7d3253650c86 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:13.313284 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747b49e8-5f20-43ce-978d-7d3253650c86 None admin] PATH_INFO: `/v3/projects/72e07920945d4b0aa18d5ce7a1213a86/users/90433f7da3c04bbfb8ffe2a3ec4d3491/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:13.316235 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747b49e8-5f20-43ce-978d-7d3253650c86 None admin] RBAC: Authorizing `identity:create_grant(project_id=72e07920945d4b0aa18d5ce7a1213a86, user_id=90433f7da3c04bbfb8ffe2a3ec4d3491, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:13.317703 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747b49e8-5f20-43ce-978d-7d3253650c86 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:13.351906 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:13.357476 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843beaa9-dfe9-419f-827e-b4c03825a027 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:13.380119 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 551/1083] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Aug 29 22:30:13 2022] PUT /identity/v3/projects/72e07920945d4b0aa18d5ce7a1213a86/users/90433f7da3c04bbfb8ffe2a3ec4d3491/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:30:13.403151 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4174e0c-a361-4ff8-810d-5fa69ebac40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:13.403836 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4174e0c-a361-4ff8-810d-5fa69ebac40f None admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:13.404107 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4174e0c-a361-4ff8-810d-5fa69ebac40f None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:13.404380 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4174e0c-a361-4ff8-810d-5fa69ebac40f None admin] PATH_INFO: `/v3/projects/ee84c3340f2641ad963655a7a212ef96/users/bdbf961e02934e88af3a72d6218f32fa/roles/6912999fb1a046b48a32d8fd29c1922d` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:13.406922 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4174e0c-a361-4ff8-810d-5fa69ebac40f None admin] RBAC: Authorizing `identity:create_grant(project_id=ee84c3340f2641ad963655a7a212ef96, user_id=bdbf961e02934e88af3a72d6218f32fa, role_id=6912999fb1a046b48a32d8fd29c1922d)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:13.408405 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4174e0c-a361-4ff8-810d-5fa69ebac40f None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:13.465676 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 533/1084] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Aug 29 22:30:13 2022] PUT /identity/v3/projects/ee84c3340f2641ad963655a7a212ef96/users/bdbf961e02934e88af3a72d6218f32fa/roles/6912999fb1a046b48a32d8fd29c1922d => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:30:15.330368 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8580e48-0f9c-4f5a-b7eb-effdfd02d1f2 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:15.330904 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8580e48-0f9c-4f5a-b7eb-effdfd02d1f2 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:15.331040 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8580e48-0f9c-4f5a-b7eb-effdfd02d1f2 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:15.332381 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 552/1085] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:30:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:30:15.337676 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2835f37-87f3-4bf9-97ca-1eb94418b53a None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:15.338080 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2835f37-87f3-4bf9-97ca-1eb94418b53a None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:15.338221 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2835f37-87f3-4bf9-97ca-1eb94418b53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:15.356966 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-b2835f37-87f3-4bf9-97ca-1eb94418b53a None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:30:15.358143 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-b2835f37-87f3-4bf9-97ca-1eb94418b53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:30:15.402026 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df3b68-efc9-4aae-8801-318b09158b66 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:15.402312 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df3b68-efc9-4aae-8801-318b09158b66 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:15.402589 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df3b68-efc9-4aae-8801-318b09158b66 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:15.403948 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 553/1086] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:30:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:30:15.409853 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa95ad-5cf6-4227-b382-705c8c042f66 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:15.410129 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa95ad-5cf6-4227-b382-705c8c042f66 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:15.410410 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa95ad-5cf6-4227-b382-705c8c042f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:15.432632 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 534/1087] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:30:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:30:15.433889 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-47fa95ad-5cf6-4227-b382-705c8c042f66 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:30:15.434385 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-47fa95ad-5cf6-4227-b382-705c8c042f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:30:15.510182 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 554/1088] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:30:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:30:15.710229 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2835f37-87f3-4bf9-97ca-1eb94418b53a None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:15.794387 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fa95ad-5cf6-4227-b382-705c8c042f66 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:15.816888 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:15.834695 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2835f37-87f3-4bf9-97ca-1eb94418b53a None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:15.869208 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:15.879003 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848b66c-15b0-47ee-a698-e78f5accb4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:15.906652 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848b66c-15b0-47ee-a698-e78f5accb4d8 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:15.906652 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848b66c-15b0-47ee-a698-e78f5accb4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:15.906652 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848b66c-15b0-47ee-a698-e78f5accb4d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:15.906652 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2848b66c-15b0-47ee-a698-e78f5accb4d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:30:15.906652 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-2848b66c-15b0-47ee-a698-e78f5accb4d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:30:15.906652 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:30:15.906652 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:30:15.906652 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:30:15.906652 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:30:15.907488 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:30:15.907488 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:30:15.907488 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:30:15.907488 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:30:15.907488 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:30:15.907488 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:30:15.907488 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:30:15.907488 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:30:15.907488 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:30:15.907488 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:30:15.907488 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:30:15.907488 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:30:15.907488 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:30:15.907488 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:30:15.907488 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:30:15.907488 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:30:15.908842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:30:15.908842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:30:15.908842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:30:15.908842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:30:15.908842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:30:15.908842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:30:15.908842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:30:15.908842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:30:15.908842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:30:15.908842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:30:15.908842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:30:15.908842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:30:15.908842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:30:15.908842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:30:15.908842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:30:15.908842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:30:15.908842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:30:15.909948 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:30:15.909948 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:30:15.909948 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:30:15.909948 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:30:15.909948 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:30:15.909948 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:30:15.909948 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 535/1089] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Aug 29 22:30:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:30:15.909948 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848b66c-15b0-47ee-a698-e78f5accb4d8 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:15.930636 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fa95ad-5cf6-4227-b382-705c8c042f66 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:15.965993 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3a5f1-86cd-469e-a96f-307182502dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:15.966543 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3a5f1-86cd-469e-a96f-307182502dbe None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:15.966822 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3a5f1-86cd-469e-a96f-307182502dbe None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:15.967088 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3a5f1-86cd-469e-a96f-307182502dbe None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:15.977266 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb3a5f1-86cd-469e-a96f-307182502dbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:30:15.982031 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-abb3a5f1-86cd-469e-a96f-307182502dbe None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 22:30:15.982031 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:30:15.982031 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:30:15.982031 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:30:15.982031 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:30:15.982031 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:30:15.982031 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:30:15.982031 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:30:15.982031 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:30:15.982031 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:30:15.982031 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:30:15.982031 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:30:15.982031 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:30:15.982031 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:30:15.982031 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:30:15.982031 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:30:15.983141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:30:15.983141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:30:15.983141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:30:15.983141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:30:15.983141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:30:15.983141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:30:15.983141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:30:15.983141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:30:15.983141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:30:15.983141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:30:15.983141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:30:15.983141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:30:15.983141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:30:15.983141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:30:15.983141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:30:15.983141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:30:15.983978 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:30:15.983978 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:30:15.983978 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:30:15.983978 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:30:15.983978 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:30:15.983978 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:30:15.983978 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:30:15.983978 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:30:15.983978 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:30:15.983978 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:30:15.983978 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 22:30:15.983978 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:30:15.983978 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 555/1090] 10.222.0.52 () {56 vars in 1120 bytes} [Mon Aug 29 22:30:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:30:15.990099 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3a5f1-86cd-469e-a96f-307182502dbe None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:16.012594 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848b66c-15b0-47ee-a698-e78f5accb4d8 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:16.049458 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d49f7b-14a7-41c4-add4-5b6f62d46867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:16.052507 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d49f7b-14a7-41c4-add4-5b6f62d46867 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:16.052507 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d49f7b-14a7-41c4-add4-5b6f62d46867 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:16.052507 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d49f7b-14a7-41c4-add4-5b6f62d46867 None admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:16.052507 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d49f7b-14a7-41c4-add4-5b6f62d46867 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:30:16.052507 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d49f7b-14a7-41c4-add4-5b6f62d46867 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:16.053243 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d49f7b-14a7-41c4-add4-5b6f62d46867 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:16.061940 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 536/1091] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:30:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:30:16.067986 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d49f7b-14a7-41c4-add4-5b6f62d46867 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:16.078609 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:16.094539 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3a5f1-86cd-469e-a96f-307182502dbe None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:16.105710 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:16.130262 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d4dc4c-3f47-4e3c-9e22-b210cb38c1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:16.131222 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4dc4c-3f47-4e3c-9e22-b210cb38c1ad None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:16.131222 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4dc4c-3f47-4e3c-9e22-b210cb38c1ad None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:16.131431 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4dc4c-3f47-4e3c-9e22-b210cb38c1ad None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:16.132041 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d4dc4c-3f47-4e3c-9e22-b210cb38c1ad None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:30:16.132493 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d4dc4c-3f47-4e3c-9e22-b210cb38c1ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:16.133914 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d4dc4c-3f47-4e3c-9e22-b210cb38c1ad None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:16.143383 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 556/1092] 10.222.0.52 () {56 vars in 1115 bytes} [Mon Aug 29 22:30:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:30:16.149104 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d4dc4c-3f47-4e3c-9e22-b210cb38c1ad None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:16.170168 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d49f7b-14a7-41c4-add4-5b6f62d46867 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:16.209072 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317e6ccc-3bd1-40b8-9f9c-ec7d0a0db6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:16.209711 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e6ccc-3bd1-40b8-9f9c-ec7d0a0db6ba None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:16.209995 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e6ccc-3bd1-40b8-9f9c-ec7d0a0db6ba None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:16.210273 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e6ccc-3bd1-40b8-9f9c-ec7d0a0db6ba None admin] PATH_INFO: `/v3/users/90433f7da3c04bbfb8ffe2a3ec4d3491` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:16.211419 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317e6ccc-3bd1-40b8-9f9c-ec7d0a0db6ba None admin] RBAC: Authorizing `identity:get_user(user_id=90433f7da3c04bbfb8ffe2a3ec4d3491)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:16.212893 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317e6ccc-3bd1-40b8-9f9c-ec7d0a0db6ba None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:16.214651 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 537/1093] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:30:16 2022] GET /identity/v3/users/90433f7da3c04bbfb8ffe2a3ec4d3491 => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:30:16.221004 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317e6ccc-3bd1-40b8-9f9c-ec7d0a0db6ba None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:16.262868 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d4dc4c-3f47-4e3c-9e22-b210cb38c1ad None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:16.297650 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a1f0c-6679-4823-9f2c-e2fde28ad00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:16.298275 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a1f0c-6679-4823-9f2c-e2fde28ad00e None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:16.298654 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a1f0c-6679-4823-9f2c-e2fde28ad00e None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:16.299534 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a1f0c-6679-4823-9f2c-e2fde28ad00e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:16.299767 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:16.310197 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a1f0c-6679-4823-9f2c-e2fde28ad00e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:16.314654 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a1f0c-6679-4823-9f2c-e2fde28ad00e None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:16.322957 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317e6ccc-3bd1-40b8-9f9c-ec7d0a0db6ba None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:16.326718 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-d72a1f0c-6679-4823-9f2c-e2fde28ad00e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:30:16.326718 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:30:16.326718 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:30:16.326718 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:30:16.326718 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:30:16.326718 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:30:16.326718 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:30:16.326718 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:30:16.326718 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:30:16.326718 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:30:16.326718 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:30:16.326718 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:30:16.326718 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:30:16.326718 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:30:16.326718 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:30:16.326718 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:30:16.327956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:30:16.327956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:30:16.327956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:30:16.327956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:30:16.327956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:30:16.327956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:30:16.327956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:30:16.327956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:30:16.327956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:30:16.327956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:30:16.327956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:30:16.327956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:30:16.327956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:30:16.327956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:30:16.327956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:30:16.327956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:30:16.327956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:30:16.328902 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:30:16.328902 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:30:16.328902 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:30:16.328902 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 557/1094] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:30:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:30:16.334292 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a1f0c-6679-4823-9f2c-e2fde28ad00e None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:16.359934 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:16.360108 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa75f67-5ed3-4ba2-a3dd-d13eaac67b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:16.363509 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa75f67-5ed3-4ba2-a3dd-d13eaac67b26 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:16.363509 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa75f67-5ed3-4ba2-a3dd-d13eaac67b26 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:16.363509 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa75f67-5ed3-4ba2-a3dd-d13eaac67b26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:16.432629 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa75f67-5ed3-4ba2-a3dd-d13eaac67b26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:16.434198 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa75f67-5ed3-4ba2-a3dd-d13eaac67b26 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:16.436531 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a1f0c-6679-4823-9f2c-e2fde28ad00e None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:16.472008 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f376e8ed-0710-48ae-bb8e-6b02decac38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:16.472538 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376e8ed-0710-48ae-bb8e-6b02decac38a None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:16.472840 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376e8ed-0710-48ae-bb8e-6b02decac38a None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:16.473110 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376e8ed-0710-48ae-bb8e-6b02decac38a None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:16.473752 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f376e8ed-0710-48ae-bb8e-6b02decac38a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:30:16.474052 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f376e8ed-0710-48ae-bb8e-6b02decac38a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:16.475517 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f376e8ed-0710-48ae-bb8e-6b02decac38a None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:16.489632 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 558/1095] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:30:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:30:16.495578 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f376e8ed-0710-48ae-bb8e-6b02decac38a None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:16.537741 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:16.558392 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 538/1096] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:30:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 22:30:16.566152 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa75f67-5ed3-4ba2-a3dd-d13eaac67b26 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:16.586581 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:16.602910 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f376e8ed-0710-48ae-bb8e-6b02decac38a None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:16.639267 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b7db67-cbd8-4d78-9886-59f52310889d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:16.640004 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b7db67-cbd8-4d78-9886-59f52310889d None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:16.640346 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b7db67-cbd8-4d78-9886-59f52310889d None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:16.640707 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b7db67-cbd8-4d78-9886-59f52310889d None admin] PATH_INFO: `/v3/users/nova` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:16.648073 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b7db67-cbd8-4d78-9886-59f52310889d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:16.649528 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b7db67-cbd8-4d78-9886-59f52310889d None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:16.661255 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-d3b7db67-cbd8-4d78-9886-59f52310889d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 22:30:16.661255 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:30:16.661255 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:30:16.661255 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:30:16.661255 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:30:16.661255 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:30:16.661255 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:30:16.661255 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:30:16.661255 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:30:16.661255 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:30:16.661255 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:30:16.661255 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:30:16.661255 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:30:16.661255 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:30:16.661255 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:30:16.661255 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:30:16.662331 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:30:16.662331 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:30:16.662331 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:30:16.662331 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:30:16.662331 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:30:16.662331 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:30:16.662331 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:30:16.662331 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:30:16.662331 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:30:16.662331 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:30:16.662331 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:30:16.662331 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:30:16.662331 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:30:16.662331 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:30:16.662331 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:30:16.662331 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:30:16.662331 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:30:16.662331 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:30:16.663313 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:30:16.663313 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:30:16.663313 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:30:16.663313 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:30:16.663313 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:30:16.663313 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:30:16.663313 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:30:16.663313 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:30:16.663313 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 22:30:16.663313 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:30:16.663313 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 559/1097] 10.222.0.52 () {56 vars in 1093 bytes} [Mon Aug 29 22:30:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:30:16.666964 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa75f67-5ed3-4ba2-a3dd-d13eaac67b26 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:16.668003 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b7db67-cbd8-4d78-9886-59f52310889d None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:16.703211 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24adb372-5b5e-49f4-8126-bcbdeb2f3c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:16.703840 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24adb372-5b5e-49f4-8126-bcbdeb2f3c17 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:16.704113 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24adb372-5b5e-49f4-8126-bcbdeb2f3c17 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:16.704386 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24adb372-5b5e-49f4-8126-bcbdeb2f3c17 None admin] PATH_INFO: `/v3/projects/72e07920945d4b0aa18d5ce7a1213a86` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:16.705752 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24adb372-5b5e-49f4-8126-bcbdeb2f3c17 None admin] RBAC: Authorizing `identity:get_project(project_id=72e07920945d4b0aa18d5ce7a1213a86)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:16.707177 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24adb372-5b5e-49f4-8126-bcbdeb2f3c17 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:16.708828 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 539/1098] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:30:16 2022] GET /identity/v3/projects/72e07920945d4b0aa18d5ce7a1213a86 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:30:16.714759 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24adb372-5b5e-49f4-8126-bcbdeb2f3c17 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:16.760478 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:16.772585 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b7db67-cbd8-4d78-9886-59f52310889d None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:16.808094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa82308-cf35-4f82-a619-a27b20c68255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:16.808691 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa82308-cf35-4f82-a619-a27b20c68255 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:16.808995 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa82308-cf35-4f82-a619-a27b20c68255 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:16.809265 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa82308-cf35-4f82-a619-a27b20c68255 None admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:16.809981 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa82308-cf35-4f82-a619-a27b20c68255 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:30:16.810284 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa82308-cf35-4f82-a619-a27b20c68255 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:16.811697 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa82308-cf35-4f82-a619-a27b20c68255 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:16.815059 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24adb372-5b5e-49f4-8126-bcbdeb2f3c17 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:16.828926 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 560/1099] 10.222.0.52 () {56 vars in 1133 bytes} [Mon Aug 29 22:30:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:30:16.834701 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa82308-cf35-4f82-a619-a27b20c68255 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:16.837364 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:16.850081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910beda-5857-4c1a-95b6-e84d3d5cc589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:16.858860 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910beda-5857-4c1a-95b6-e84d3d5cc589 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:16.858860 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910beda-5857-4c1a-95b6-e84d3d5cc589 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:16.858860 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910beda-5857-4c1a-95b6-e84d3d5cc589 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:16.858860 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2910beda-5857-4c1a-95b6-e84d3d5cc589 None admin] RBAC: Adding query filter params (role.id=882f35bd847a406eb2222aeafc517a46, scope.project.id=72e07920945d4b0aa18d5ce7a1213a86, user.id=90433f7da3c04bbfb8ffe2a3ec4d3491) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:30:16.858860 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2910beda-5857-4c1a-95b6-e84d3d5cc589 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:16.858860 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2910beda-5857-4c1a-95b6-e84d3d5cc589 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:16.859351 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 540/1100] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:30:16 2022] GET /identity/v3/role_assignments?user.id=90433f7da3c04bbfb8ffe2a3ec4d3491&scope.project.id=72e07920945d4b0aa18d5ce7a1213a86&role.id=882f35bd847a406eb2222aeafc517a46 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:30:16.940405 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa82308-cf35-4f82-a619-a27b20c68255 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:16.976625 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649ef734-785a-4ef9-aa5d-c76272af3182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:16.977264 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ef734-785a-4ef9-aa5d-c76272af3182 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:16.977538 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ef734-785a-4ef9-aa5d-c76272af3182 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:16.977805 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ef734-785a-4ef9-aa5d-c76272af3182 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:17.049228 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649ef734-785a-4ef9-aa5d-c76272af3182 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:17.050617 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649ef734-785a-4ef9-aa5d-c76272af3182 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:17.052800 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:17.208993 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 561/1101] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:30:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 22:30:17.210210 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910beda-5857-4c1a-95b6-e84d3d5cc589 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:17.312297 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910beda-5857-4c1a-95b6-e84d3d5cc589 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:17.321570 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:17.347840 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b6681-063d-4f32-9cd7-b0d487837731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:17.348465 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b6681-063d-4f32-9cd7-b0d487837731 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:17.348767 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b6681-063d-4f32-9cd7-b0d487837731 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:17.349043 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b6681-063d-4f32-9cd7-b0d487837731 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:17.359853 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094b6681-063d-4f32-9cd7-b0d487837731 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:17.361905 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094b6681-063d-4f32-9cd7-b0d487837731 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:17.375964 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-094b6681-063d-4f32-9cd7-b0d487837731 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:30:17.375964 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:30:17.375964 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:30:17.375964 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:30:17.375964 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:30:17.375964 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:30:17.375964 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:30:17.375964 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:30:17.375964 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:30:17.375964 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:30:17.375964 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:30:17.375964 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:30:17.375964 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:30:17.375964 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:30:17.375964 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:30:17.375964 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:30:17.377271 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:30:17.377271 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:30:17.377271 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:30:17.377271 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:30:17.377271 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:30:17.377271 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:30:17.377271 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:30:17.377271 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:30:17.377271 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:30:17.377271 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:30:17.377271 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:30:17.377271 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:30:17.377271 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:30:17.377271 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:30:17.377271 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:30:17.377271 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:30:17.377271 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:30:17.378173 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:30:17.378173 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:30:17.378173 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:30:17.378173 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 541/1102] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:30:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:30:17.383443 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649ef734-785a-4ef9-aa5d-c76272af3182 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:17.487881 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649ef734-785a-4ef9-aa5d-c76272af3182 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:17.489485 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:17.522572 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a46ba3a-f762-420f-8a6e-00d5a3139551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:17.523182 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a46ba3a-f762-420f-8a6e-00d5a3139551 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:17.523462 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a46ba3a-f762-420f-8a6e-00d5a3139551 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:17.523737 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a46ba3a-f762-420f-8a6e-00d5a3139551 None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:17.524376 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a46ba3a-f762-420f-8a6e-00d5a3139551 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:30:17.524682 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a46ba3a-f762-420f-8a6e-00d5a3139551 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:17.526089 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a46ba3a-f762-420f-8a6e-00d5a3139551 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:17.540544 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 562/1103] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:30:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:17.546040 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b6681-063d-4f32-9cd7-b0d487837731 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:17.648606 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b6681-063d-4f32-9cd7-b0d487837731 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:17.682500 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc6618-b304-4c23-8f2d-7f410776aaba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:17.683111 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc6618-b304-4c23-8f2d-7f410776aaba None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:17.683385 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc6618-b304-4c23-8f2d-7f410776aaba None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:17.683650 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc6618-b304-4c23-8f2d-7f410776aaba None admin] PATH_INFO: `/v3/projects/service` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:17.690235 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbc6618-b304-4c23-8f2d-7f410776aaba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:17.691660 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbc6618-b304-4c23-8f2d-7f410776aaba None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:17.703791 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-9bbc6618-b304-4c23-8f2d-7f410776aaba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:30:17.703791 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:30:17.703791 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:30:17.703791 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:30:17.703791 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:30:17.703791 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:30:17.703791 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:30:17.703791 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:30:17.703791 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:30:17.703791 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:30:17.703791 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:30:17.703791 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:30:17.703791 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:30:17.703791 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:30:17.703791 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:30:17.704664 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:30:17.704664 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:30:17.704664 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:30:17.704664 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:30:17.704664 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:30:17.704664 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:30:17.704664 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:30:17.704664 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:30:17.704664 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:30:17.704664 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:30:17.704664 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:30:17.704664 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:30:17.704664 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:30:17.704664 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:30:17.704664 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:30:17.704664 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:30:17.704664 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:30:17.705883 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:30:17.705883 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:30:17.705883 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:30:17.705883 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:30:17.705883 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:30:17.705883 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:30:17.705883 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:30:17.705883 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:30:17.705883 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 542/1104] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:30:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:30:17.710791 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a46ba3a-f762-420f-8a6e-00d5a3139551 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:17.820348 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a46ba3a-f762-420f-8a6e-00d5a3139551 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:17.853782 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb67fd90-5f98-4e9c-b63d-7855e6a843f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:17.854378 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb67fd90-5f98-4e9c-b63d-7855e6a843f5 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:17.854662 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb67fd90-5f98-4e9c-b63d-7855e6a843f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:17.854937 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb67fd90-5f98-4e9c-b63d-7855e6a843f5 None admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:17.855620 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb67fd90-5f98-4e9c-b63d-7855e6a843f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:30:17.855927 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb67fd90-5f98-4e9c-b63d-7855e6a843f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:17.857437 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb67fd90-5f98-4e9c-b63d-7855e6a843f5 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:17.869845 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 563/1105] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Aug 29 22:30:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:17.875934 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc6618-b304-4c23-8f2d-7f410776aaba None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:17.899044 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:17.977954 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc6618-b304-4c23-8f2d-7f410776aaba None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:18.012320 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c92fa-840d-41ed-86c6-115239801981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:18.012951 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c92fa-840d-41ed-86c6-115239801981 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:18.013233 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c92fa-840d-41ed-86c6-115239801981 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:18.013506 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c92fa-840d-41ed-86c6-115239801981 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:18.014225 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c92fa-840d-41ed-86c6-115239801981 None admin] RBAC: Adding query filter params (role.id=6912999fb1a046b48a32d8fd29c1922d, scope.project.id=ee84c3340f2641ad963655a7a212ef96, user.id=bdbf961e02934e88af3a72d6218f32fa) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:30:18.014535 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c92fa-840d-41ed-86c6-115239801981 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:18.015975 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c92fa-840d-41ed-86c6-115239801981 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:18.023043 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 543/1106] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:30:17 2022] GET /identity/v3/role_assignments?user.id=bdbf961e02934e88af3a72d6218f32fa&scope.project.id=ee84c3340f2641ad963655a7a212ef96&role.id=6912999fb1a046b48a32d8fd29c1922d => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:18.906179 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b252c-b36d-437d-a4cf-f057a1f73207 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:18.906663 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b252c-b36d-437d-a4cf-f057a1f73207 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:18.906663 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b252c-b36d-437d-a4cf-f057a1f73207 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:18.908560 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 564/1107] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:30:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:30:18.913995 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06cba4-454d-4329-b739-47188746760e None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:18.914438 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06cba4-454d-4329-b739-47188746760e None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:18.914530 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06cba4-454d-4329-b739-47188746760e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:18.933740 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-ba06cba4-454d-4329-b739-47188746760e None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:30:18.934938 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-ba06cba4-454d-4329-b739-47188746760e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:30:19.004566 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 544/1108] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:30:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:30:19.279413 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7b252c-b36d-437d-a4cf-f057a1f73207 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:19.306407 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:19.383848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7b252c-b36d-437d-a4cf-f057a1f73207 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:19.417735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ac665-c9ed-4fb2-9a96-aa59c51d3f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:19.418353 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ac665-c9ed-4fb2-9a96-aa59c51d3f30 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:19.418632 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ac665-c9ed-4fb2-9a96-aa59c51d3f30 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:19.418930 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ac665-c9ed-4fb2-9a96-aa59c51d3f30 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:19.420029 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7ac665-c9ed-4fb2-9a96-aa59c51d3f30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:19.421479 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7ac665-c9ed-4fb2-9a96-aa59c51d3f30 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:19.423044 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 565/1109] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:30:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:19.429184 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba06cba4-454d-4329-b739-47188746760e None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:19.443982 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:19.529471 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba06cba4-454d-4329-b739-47188746760e None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:19.564509 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f6758-c98b-4c96-95e8-7fc7b1e21291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:19.565155 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f6758-c98b-4c96-95e8-7fc7b1e21291 None admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:19.565437 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f6758-c98b-4c96-95e8-7fc7b1e21291 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:19.565713 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f6758-c98b-4c96-95e8-7fc7b1e21291 None admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:19.566358 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508f6758-c98b-4c96-95e8-7fc7b1e21291 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:19.567781 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508f6758-c98b-4c96-95e8-7fc7b1e21291 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:19.698842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 545/1110] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Aug 29 22:30:19 2022] POST /identity/v3/users => generated 286 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:30:21.683145 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c10bc4-25d4-4e32-acf9-8c03d968f213 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:21.683695 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c10bc4-25d4-4e32-acf9-8c03d968f213 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:21.683695 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c10bc4-25d4-4e32-acf9-8c03d968f213 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:21.685296 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 566/1111] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:30:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:30:21.691530 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5121c8e-d6c3-4f48-a077-fe45d3a426ad None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:21.691927 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5121c8e-d6c3-4f48-a077-fe45d3a426ad None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:21.692082 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5121c8e-d6c3-4f48-a077-fe45d3a426ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:21.710720 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-e5121c8e-d6c3-4f48-a077-fe45d3a426ad None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:30:21.711939 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-e5121c8e-d6c3-4f48-a077-fe45d3a426ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:30:21.778921 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:21.783827 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 546/1112] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:30:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:30:22.062189 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c10bc4-25d4-4e32-acf9-8c03d968f213 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:22.159352 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:22.169485 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c10bc4-25d4-4e32-acf9-8c03d968f213 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:22.204358 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141fe3f3-fbe7-43e8-b356-47ba5a50f321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:22.205173 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141fe3f3-fbe7-43e8-b356-47ba5a50f321 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:22.205449 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141fe3f3-fbe7-43e8-b356-47ba5a50f321 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:22.205722 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141fe3f3-fbe7-43e8-b356-47ba5a50f321 None admin] PATH_INFO: `/v3/roles/a6d1677ebe3a4355be5d67c5c78d83c7` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:22.207155 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141fe3f3-fbe7-43e8-b356-47ba5a50f321 None admin] RBAC: Authorizing `identity:get_role(role_id=a6d1677ebe3a4355be5d67c5c78d83c7)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:22.208555 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141fe3f3-fbe7-43e8-b356-47ba5a50f321 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:22.209913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 567/1113] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:30:22 2022] GET /identity/v3/roles/a6d1677ebe3a4355be5d67c5c78d83c7 => generated 223 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:22.215754 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5121c8e-d6c3-4f48-a077-fe45d3a426ad None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:22.315867 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5121c8e-d6c3-4f48-a077-fe45d3a426ad None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:22.349405 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b640e0-0fa6-472a-86de-c8af75e7460e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:22.350037 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b640e0-0fa6-472a-86de-c8af75e7460e None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:22.350318 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b640e0-0fa6-472a-86de-c8af75e7460e None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:22.350592 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b640e0-0fa6-472a-86de-c8af75e7460e None admin] PATH_INFO: `/v3/users/ceb7acc1049d4d869e0509ce732ed852` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:22.364848 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b640e0-0fa6-472a-86de-c8af75e7460e None admin] RBAC: Authorizing `identity:get_user(user_id=ceb7acc1049d4d869e0509ce732ed852)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:22.366306 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b640e0-0fa6-472a-86de-c8af75e7460e None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:22.367930 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 547/1114] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:30:22 2022] GET /identity/v3/users/ceb7acc1049d4d869e0509ce732ed852 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:22.374138 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141fe3f3-fbe7-43e8-b356-47ba5a50f321 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:22.472447 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141fe3f3-fbe7-43e8-b356-47ba5a50f321 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:22.505380 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491489db-e287-482a-8b95-cf8559a703af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:22.505964 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491489db-e287-482a-8b95-cf8559a703af None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:22.506242 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491489db-e287-482a-8b95-cf8559a703af None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:22.506511 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491489db-e287-482a-8b95-cf8559a703af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:22.532619 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:22.575990 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491489db-e287-482a-8b95-cf8559a703af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:22.577408 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491489db-e287-482a-8b95-cf8559a703af None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:22.677419 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 568/1115] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:30:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 22:30:22.683386 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b640e0-0fa6-472a-86de-c8af75e7460e None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:22.741789 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:22.783627 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b640e0-0fa6-472a-86de-c8af75e7460e None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:22.817791 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480a36d-aa18-4ec8-9ee1-32480e1eaeb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:22.818411 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480a36d-aa18-4ec8-9ee1-32480e1eaeb5 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:22.818685 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480a36d-aa18-4ec8-9ee1-32480e1eaeb5 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:22.818955 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480a36d-aa18-4ec8-9ee1-32480e1eaeb5 None admin] PATH_INFO: `/v3/projects/72e07920945d4b0aa18d5ce7a1213a86` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:22.820039 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b480a36d-aa18-4ec8-9ee1-32480e1eaeb5 None admin] RBAC: Authorizing `identity:get_project(project_id=72e07920945d4b0aa18d5ce7a1213a86)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:22.821495 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b480a36d-aa18-4ec8-9ee1-32480e1eaeb5 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:22.823100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 548/1116] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:30:22 2022] GET /identity/v3/projects/72e07920945d4b0aa18d5ce7a1213a86 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:22.829453 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491489db-e287-482a-8b95-cf8559a703af None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:22.889057 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:22.930804 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491489db-e287-482a-8b95-cf8559a703af None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:22.965047 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5581a726-f4c2-4e2e-b070-8746d96131c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:22.965641 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581a726-f4c2-4e2e-b070-8746d96131c7 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:22.965923 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581a726-f4c2-4e2e-b070-8746d96131c7 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:22.966277 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581a726-f4c2-4e2e-b070-8746d96131c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:22.966995 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5581a726-f4c2-4e2e-b070-8746d96131c7 None admin] RBAC: Adding query filter params (role.id=a6d1677ebe3a4355be5d67c5c78d83c7, scope.project.id=72e07920945d4b0aa18d5ce7a1213a86, user.id=ceb7acc1049d4d869e0509ce732ed852) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:30:22.967303 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5581a726-f4c2-4e2e-b070-8746d96131c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:22.968762 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5581a726-f4c2-4e2e-b070-8746d96131c7 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:22.974443 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 569/1117] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:30:22 2022] GET /identity/v3/role_assignments?user.id=ceb7acc1049d4d869e0509ce732ed852&scope.project.id=72e07920945d4b0aa18d5ce7a1213a86&role.id=a6d1677ebe3a4355be5d67c5c78d83c7 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:24.878003 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f62ce5b-5018-4ef0-a75e-1c81fd42e62e None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:24.878501 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f62ce5b-5018-4ef0-a75e-1c81fd42e62e None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:24.878501 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f62ce5b-5018-4ef0-a75e-1c81fd42e62e None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:24.879804 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 549/1118] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:30:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:30:24.884977 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70c62f-719d-4a9f-b1a0-dc767c724d59 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:24.885411 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70c62f-719d-4a9f-b1a0-dc767c724d59 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:24.885548 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70c62f-719d-4a9f-b1a0-dc767c724d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:24.911200 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-1b70c62f-719d-4a9f-b1a0-dc767c724d59 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:30:24.912372 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-1b70c62f-719d-4a9f-b1a0-dc767c724d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:30:24.992002 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 570/1119] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:30:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:30:25.264676 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f62ce5b-5018-4ef0-a75e-1c81fd42e62e None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:25.373962 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f62ce5b-5018-4ef0-a75e-1c81fd42e62e None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:25.399642 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:25.408281 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a27d85b-1185-4a08-964a-1605419cb1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:25.409136 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a27d85b-1185-4a08-964a-1605419cb1ae None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:25.415156 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a27d85b-1185-4a08-964a-1605419cb1ae None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:25.415156 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a27d85b-1185-4a08-964a-1605419cb1ae None admin] PATH_INFO: `/v3/roles/a6d1677ebe3a4355be5d67c5c78d83c7` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:25.415156 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a27d85b-1185-4a08-964a-1605419cb1ae None admin] RBAC: Authorizing `identity:get_role(role_id=a6d1677ebe3a4355be5d67c5c78d83c7)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:25.415156 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a27d85b-1185-4a08-964a-1605419cb1ae None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:25.415156 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 550/1120] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:30:25 2022] GET /identity/v3/roles/a6d1677ebe3a4355be5d67c5c78d83c7 => generated 223 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:25.419596 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70c62f-719d-4a9f-b1a0-dc767c724d59 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:25.472989 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:25.518887 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70c62f-719d-4a9f-b1a0-dc767c724d59 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:25.553649 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b26d7f-d8ff-43d2-9902-fc25075bdfb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:25.554275 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b26d7f-d8ff-43d2-9902-fc25075bdfb9 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:25.554550 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b26d7f-d8ff-43d2-9902-fc25075bdfb9 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:25.554820 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b26d7f-d8ff-43d2-9902-fc25075bdfb9 None admin] PATH_INFO: `/v3/users/ceb7acc1049d4d869e0509ce732ed852` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:25.555916 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b26d7f-d8ff-43d2-9902-fc25075bdfb9 None admin] RBAC: Authorizing `identity:get_user(user_id=ceb7acc1049d4d869e0509ce732ed852)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:25.557379 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b26d7f-d8ff-43d2-9902-fc25075bdfb9 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:25.558935 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 571/1121] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:30:25 2022] GET /identity/v3/users/ceb7acc1049d4d869e0509ce732ed852 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:25.564945 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a27d85b-1185-4a08-964a-1605419cb1ae None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:25.665693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a27d85b-1185-4a08-964a-1605419cb1ae None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:25.699828 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1da919-797b-49fd-8e02-05a966c5b331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:25.700443 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1da919-797b-49fd-8e02-05a966c5b331 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:25.700741 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1da919-797b-49fd-8e02-05a966c5b331 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:25.701011 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1da919-797b-49fd-8e02-05a966c5b331 None admin] PATH_INFO: `/v3/projects/72e07920945d4b0aa18d5ce7a1213a86` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:25.702096 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1da919-797b-49fd-8e02-05a966c5b331 None admin] RBAC: Authorizing `identity:get_project(project_id=72e07920945d4b0aa18d5ce7a1213a86)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:25.703510 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1da919-797b-49fd-8e02-05a966c5b331 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:25.705848 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 551/1122] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:30:25 2022] GET /identity/v3/projects/72e07920945d4b0aa18d5ce7a1213a86 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:25.711908 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b26d7f-d8ff-43d2-9902-fc25075bdfb9 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:25.811166 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b26d7f-d8ff-43d2-9902-fc25075bdfb9 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:25.836227 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:25.849833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afafa6ee-96eb-416b-b32c-1c7a8d963e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:25.850493 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afafa6ee-96eb-416b-b32c-1c7a8d963e88 None admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:25.852815 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afafa6ee-96eb-416b-b32c-1c7a8d963e88 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:25.852815 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afafa6ee-96eb-416b-b32c-1c7a8d963e88 None admin] PATH_INFO: `/v3/projects/72e07920945d4b0aa18d5ce7a1213a86/users/ceb7acc1049d4d869e0509ce732ed852/roles/a6d1677ebe3a4355be5d67c5c78d83c7` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:25.853180 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afafa6ee-96eb-416b-b32c-1c7a8d963e88 None admin] RBAC: Authorizing `identity:create_grant(project_id=72e07920945d4b0aa18d5ce7a1213a86, user_id=ceb7acc1049d4d869e0509ce732ed852, role_id=a6d1677ebe3a4355be5d67c5c78d83c7)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:25.854600 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afafa6ee-96eb-416b-b32c-1c7a8d963e88 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:25.950040 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 572/1123] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Aug 29 22:30:25 2022] PUT /identity/v3/projects/72e07920945d4b0aa18d5ce7a1213a86/users/ceb7acc1049d4d869e0509ce732ed852/roles/a6d1677ebe3a4355be5d67c5c78d83c7 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:30:27.883440 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1824cb8-a48f-46e5-8d19-830837e8ce4d None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:27.884049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1824cb8-a48f-46e5-8d19-830837e8ce4d None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:27.884049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1824cb8-a48f-46e5-8d19-830837e8ce4d None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:27.885282 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 552/1124] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:30:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:30:27.890955 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c24cc8-cba5-4049-b86b-817c0308ad25 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:27.891457 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c24cc8-cba5-4049-b86b-817c0308ad25 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:27.891550 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c24cc8-cba5-4049-b86b-817c0308ad25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:27.910133 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-41c24cc8-cba5-4049-b86b-817c0308ad25 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:30:27.911591 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-41c24cc8-cba5-4049-b86b-817c0308ad25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:30:27.983121 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 573/1125] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:30:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:30:28.272926 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1824cb8-a48f-46e5-8d19-830837e8ce4d None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:28.337694 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:28.377476 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1824cb8-a48f-46e5-8d19-830837e8ce4d None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:28.411485 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5994e88b-6db1-47f0-b0fc-388045a7a490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:28.412118 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994e88b-6db1-47f0-b0fc-388045a7a490 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:28.412394 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994e88b-6db1-47f0-b0fc-388045a7a490 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:28.412661 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994e88b-6db1-47f0-b0fc-388045a7a490 None admin] PATH_INFO: `/v3/roles/a6d1677ebe3a4355be5d67c5c78d83c7` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:28.414067 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5994e88b-6db1-47f0-b0fc-388045a7a490 None admin] RBAC: Authorizing `identity:get_role(role_id=a6d1677ebe3a4355be5d67c5c78d83c7)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:28.415425 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5994e88b-6db1-47f0-b0fc-388045a7a490 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:28.416752 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 553/1126] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:30:28 2022] GET /identity/v3/roles/a6d1677ebe3a4355be5d67c5c78d83c7 => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:28.423985 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c24cc8-cba5-4049-b86b-817c0308ad25 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:28.525350 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c24cc8-cba5-4049-b86b-817c0308ad25 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:28.540901 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:28.564890 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f587fba-c0a3-4d4d-b595-752192cb6c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:28.565514 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f587fba-c0a3-4d4d-b595-752192cb6c04 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:28.565791 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f587fba-c0a3-4d4d-b595-752192cb6c04 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:28.566146 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f587fba-c0a3-4d4d-b595-752192cb6c04 None admin] PATH_INFO: `/v3/users/ceb7acc1049d4d869e0509ce732ed852` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:28.567254 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f587fba-c0a3-4d4d-b595-752192cb6c04 None admin] RBAC: Authorizing `identity:get_user(user_id=ceb7acc1049d4d869e0509ce732ed852)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:28.569140 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f587fba-c0a3-4d4d-b595-752192cb6c04 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:28.571617 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 574/1127] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:30:28 2022] GET /identity/v3/users/ceb7acc1049d4d869e0509ce732ed852 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:28.577474 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5994e88b-6db1-47f0-b0fc-388045a7a490 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:28.691944 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5994e88b-6db1-47f0-b0fc-388045a7a490 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:28.721858 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:28.727536 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584f15b4-c4d3-4576-ba40-7a199be8deaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:28.728137 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f15b4-c4d3-4576-ba40-7a199be8deaf None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:28.728432 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f15b4-c4d3-4576-ba40-7a199be8deaf None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:28.728693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f15b4-c4d3-4576-ba40-7a199be8deaf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:28.797708 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584f15b4-c4d3-4576-ba40-7a199be8deaf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:28.799140 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584f15b4-c4d3-4576-ba40-7a199be8deaf None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:28.903794 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 554/1128] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:30:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 22:30:28.909940 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f587fba-c0a3-4d4d-b595-752192cb6c04 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:29.041284 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f587fba-c0a3-4d4d-b595-752192cb6c04 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:29.090380 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ea5735-7a8e-444c-a921-8443f4bcf1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:29.091204 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea5735-7a8e-444c-a921-8443f4bcf1d8 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:29.091614 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea5735-7a8e-444c-a921-8443f4bcf1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:29.092014 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea5735-7a8e-444c-a921-8443f4bcf1d8 None admin] PATH_INFO: `/v3/projects/72e07920945d4b0aa18d5ce7a1213a86` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:29.093598 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ea5735-7a8e-444c-a921-8443f4bcf1d8 None admin] RBAC: Authorizing `identity:get_project(project_id=72e07920945d4b0aa18d5ce7a1213a86)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:29.095913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ea5735-7a8e-444c-a921-8443f4bcf1d8 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:29.098426 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 575/1129] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:30:28 2022] GET /identity/v3/projects/72e07920945d4b0aa18d5ce7a1213a86 => generated 311 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:29.104596 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584f15b4-c4d3-4576-ba40-7a199be8deaf None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:29.135096 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:29.208710 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584f15b4-c4d3-4576-ba40-7a199be8deaf None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:29.242666 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a928c324-6c4a-4735-b170-563d8fdafc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:29.243264 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a928c324-6c4a-4735-b170-563d8fdafc7a None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:29.243542 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a928c324-6c4a-4735-b170-563d8fdafc7a None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:29.243813 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a928c324-6c4a-4735-b170-563d8fdafc7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:29.244522 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a928c324-6c4a-4735-b170-563d8fdafc7a None admin] RBAC: Adding query filter params (role.id=a6d1677ebe3a4355be5d67c5c78d83c7, scope.project.id=72e07920945d4b0aa18d5ce7a1213a86, user.id=ceb7acc1049d4d869e0509ce732ed852) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:30:29.244856 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a928c324-6c4a-4735-b170-563d8fdafc7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:29.246288 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a928c324-6c4a-4735-b170-563d8fdafc7a None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:29.252527 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 555/1130] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:30:29 2022] GET /identity/v3/role_assignments?user.id=ceb7acc1049d4d869e0509ce732ed852&scope.project.id=72e07920945d4b0aa18d5ce7a1213a86&role.id=a6d1677ebe3a4355be5d67c5c78d83c7 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:31.192332 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e05ae-ca92-4477-90e5-8ec6f162932e None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:31.192865 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e05ae-ca92-4477-90e5-8ec6f162932e None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:31.192973 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e05ae-ca92-4477-90e5-8ec6f162932e None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:31.194245 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 576/1131] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:30:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:30:31.199605 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44331b-d1d5-4f08-ab42-f73ad9e4df75 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:31.200020 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44331b-d1d5-4f08-ab42-f73ad9e4df75 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:31.200148 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44331b-d1d5-4f08-ab42-f73ad9e4df75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:31.218211 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-fc44331b-d1d5-4f08-ab42-f73ad9e4df75 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:30:31.219380 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-fc44331b-d1d5-4f08-ab42-f73ad9e4df75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:30:31.292580 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 556/1132] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:30:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:30:31.562822 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26e05ae-ca92-4477-90e5-8ec6f162932e None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:31.611686 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:31.666416 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26e05ae-ca92-4477-90e5-8ec6f162932e None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:31.701741 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb965dc-d5c9-4f21-a34c-959d3f4ff39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:31.702361 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb965dc-d5c9-4f21-a34c-959d3f4ff39c None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:31.702633 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb965dc-d5c9-4f21-a34c-959d3f4ff39c None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:31.703330 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb965dc-d5c9-4f21-a34c-959d3f4ff39c None admin] PATH_INFO: `/v3/domains/default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:31.705208 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb965dc-d5c9-4f21-a34c-959d3f4ff39c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:31.706818 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb965dc-d5c9-4f21-a34c-959d3f4ff39c None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:31.708390 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 577/1133] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:30:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:31.714382 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44331b-d1d5-4f08-ab42-f73ad9e4df75 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:31.744038 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:31.828231 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44331b-d1d5-4f08-ab42-f73ad9e4df75 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:31.865133 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea3a41-0142-46d6-a1c7-152d983ba5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:31.865726 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea3a41-0142-46d6-a1c7-152d983ba5b0 None admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:31.866002 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea3a41-0142-46d6-a1c7-152d983ba5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:31.866273 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea3a41-0142-46d6-a1c7-152d983ba5b0 None admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:31.866920 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abea3a41-0142-46d6-a1c7-152d983ba5b0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:31.868341 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abea3a41-0142-46d6-a1c7-152d983ba5b0 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:31.969114 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 557/1134] 10.222.0.52 () {60 vars in 1141 bytes} [Mon Aug 29 22:30:31 2022] POST /identity/v3/projects => generated 311 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:30:33.994306 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c849c-79b1-4a20-abab-dfd8fa73f8fb None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:33.994921 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c849c-79b1-4a20-abab-dfd8fa73f8fb None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:33.994921 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c849c-79b1-4a20-abab-dfd8fa73f8fb None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:33.996212 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 578/1135] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:30:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:30:34.002089 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a56dffc-dfb8-4292-ba93-0ac380e162bb None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:34.002503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a56dffc-dfb8-4292-ba93-0ac380e162bb None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:34.002637 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a56dffc-dfb8-4292-ba93-0ac380e162bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:34.020682 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-3a56dffc-dfb8-4292-ba93-0ac380e162bb None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:30:34.021902 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-3a56dffc-dfb8-4292-ba93-0ac380e162bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:30:34.087979 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 558/1136] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:30:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:30:34.354325 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3c849c-79b1-4a20-abab-dfd8fa73f8fb None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:34.462359 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3c849c-79b1-4a20-abab-dfd8fa73f8fb None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:34.477670 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:34.501138 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99e791-c49e-4b1d-a262-fd29a56ebd78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:34.502389 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99e791-c49e-4b1d-a262-fd29a56ebd78 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:34.502667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99e791-c49e-4b1d-a262-fd29a56ebd78 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:34.502947 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99e791-c49e-4b1d-a262-fd29a56ebd78 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:34.504133 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd99e791-c49e-4b1d-a262-fd29a56ebd78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:34.505620 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd99e791-c49e-4b1d-a262-fd29a56ebd78 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:34.507227 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 579/1137] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:30:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:34.513242 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a56dffc-dfb8-4292-ba93-0ac380e162bb None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:34.522121 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:34.618192 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a56dffc-dfb8-4292-ba93-0ac380e162bb None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:34.652711 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f51ac8-0188-45c9-86b5-d7423234dc81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:34.653333 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f51ac8-0188-45c9-86b5-d7423234dc81 None admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:34.653615 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f51ac8-0188-45c9-86b5-d7423234dc81 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:34.653887 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f51ac8-0188-45c9-86b5-d7423234dc81 None admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:34.654536 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f51ac8-0188-45c9-86b5-d7423234dc81 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:34.656024 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f51ac8-0188-45c9-86b5-d7423234dc81 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:34.739677 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 559/1138] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Aug 29 22:30:34 2022] POST /identity/v3/users => generated 286 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:30:36.883040 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33e2ed-c09c-4b34-94ee-d7d8dc9cef86 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:36.883809 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33e2ed-c09c-4b34-94ee-d7d8dc9cef86 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:36.883809 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33e2ed-c09c-4b34-94ee-d7d8dc9cef86 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:36.884844 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 580/1139] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:30:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:30:36.890300 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b51e86-f7bd-4fd2-b777-83919513d123 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:36.890848 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b51e86-f7bd-4fd2-b777-83919513d123 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:36.891124 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b51e86-f7bd-4fd2-b777-83919513d123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:36.917572 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-26b51e86-f7bd-4fd2-b777-83919513d123 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:30:36.919333 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-26b51e86-f7bd-4fd2-b777-83919513d123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:30:36.999143 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:37.013971 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 560/1140] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:30:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:30:37.292319 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33e2ed-c09c-4b34-94ee-d7d8dc9cef86 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:37.418592 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33e2ed-c09c-4b34-94ee-d7d8dc9cef86 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:37.455764 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b30143-04b5-4ace-8ff8-e5859db13e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:37.456415 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b30143-04b5-4ace-8ff8-e5859db13e65 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:37.456696 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b30143-04b5-4ace-8ff8-e5859db13e65 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:37.457000 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b30143-04b5-4ace-8ff8-e5859db13e65 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:37.474714 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b30143-04b5-4ace-8ff8-e5859db13e65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:30:37.479941 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-64b30143-04b5-4ace-8ff8-e5859db13e65 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:30:37.479941 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:30:37.479941 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:30:37.479941 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:30:37.479941 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:30:37.479941 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:30:37.479941 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:30:37.479941 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:30:37.479941 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:30:37.479941 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:30:37.479941 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:30:37.479941 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:30:37.479941 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:30:37.479941 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:30:37.479941 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:30:37.479941 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:30:37.481269 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:30:37.481269 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:30:37.481269 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:30:37.481269 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:30:37.481269 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:30:37.481269 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:30:37.481269 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:30:37.481269 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:30:37.481269 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:30:37.481269 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:30:37.481269 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:30:37.481269 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:30:37.481269 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:30:37.481269 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:30:37.481269 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:30:37.481269 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:30:37.482411 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:30:37.482411 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:30:37.482411 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:30:37.482411 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:30:37.482411 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:30:37.482411 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:30:37.482411 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:30:37.482411 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:30:37.482411 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:30:37.482411 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:30:37.482411 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:30:37.482411 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:30:37.482411 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 581/1141] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Aug 29 22:30:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:30:37.488606 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b51e86-f7bd-4fd2-b777-83919513d123 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:37.593835 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b51e86-f7bd-4fd2-b777-83919513d123 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:37.631831 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f43f5-c078-41f6-b7b0-4dd54d5ab741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:37.632434 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f43f5-c078-41f6-b7b0-4dd54d5ab741 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:37.632708 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f43f5-c078-41f6-b7b0-4dd54d5ab741 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:37.633074 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f43f5-c078-41f6-b7b0-4dd54d5ab741 None admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:37.633737 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02f43f5-c078-41f6-b7b0-4dd54d5ab741 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:30:37.634035 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02f43f5-c078-41f6-b7b0-4dd54d5ab741 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:37.635434 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02f43f5-c078-41f6-b7b0-4dd54d5ab741 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:37.644526 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 561/1142] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:30:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:37.650452 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b30143-04b5-4ace-8ff8-e5859db13e65 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:37.696494 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:37.755120 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b30143-04b5-4ace-8ff8-e5859db13e65 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:37.791526 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3bbc8c-1366-4d62-b892-40c3932d97bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:37.792158 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3bbc8c-1366-4d62-b892-40c3932d97bf None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:37.792438 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3bbc8c-1366-4d62-b892-40c3932d97bf None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:37.792711 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3bbc8c-1366-4d62-b892-40c3932d97bf None admin] PATH_INFO: `/v3/users/b2f56c180d564d26a95d849fbebc9803` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:37.806644 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3bbc8c-1366-4d62-b892-40c3932d97bf None admin] RBAC: Authorizing `identity:get_user(user_id=b2f56c180d564d26a95d849fbebc9803)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:37.808079 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3bbc8c-1366-4d62-b892-40c3932d97bf None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:37.809781 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 582/1143] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:30:37 2022] GET /identity/v3/users/b2f56c180d564d26a95d849fbebc9803 => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:37.816559 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f43f5-c078-41f6-b7b0-4dd54d5ab741 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:37.877444 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:37.922210 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f43f5-c078-41f6-b7b0-4dd54d5ab741 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:37.957194 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812f7521-0145-4f4e-860c-f67ff5ea6d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:37.957797 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f7521-0145-4f4e-860c-f67ff5ea6d91 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:37.958075 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f7521-0145-4f4e-860c-f67ff5ea6d91 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:37.958346 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f7521-0145-4f4e-860c-f67ff5ea6d91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:38.036300 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812f7521-0145-4f4e-860c-f67ff5ea6d91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:38.037789 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812f7521-0145-4f4e-860c-f67ff5ea6d91 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:38.091535 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:38.136422 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 562/1144] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:30:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 22:30:38.143466 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3bbc8c-1366-4d62-b892-40c3932d97bf None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:38.243270 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:38.246227 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3bbc8c-1366-4d62-b892-40c3932d97bf None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:38.282480 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91481b1-4eaa-48bd-a6fd-de1da1f0c98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:38.283688 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91481b1-4eaa-48bd-a6fd-de1da1f0c98e None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:38.283969 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91481b1-4eaa-48bd-a6fd-de1da1f0c98e None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:38.284240 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91481b1-4eaa-48bd-a6fd-de1da1f0c98e None admin] PATH_INFO: `/v3/projects/742d51a6036b48198fb1954a1e1f42cc` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:38.285373 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91481b1-4eaa-48bd-a6fd-de1da1f0c98e None admin] RBAC: Authorizing `identity:get_project(project_id=742d51a6036b48198fb1954a1e1f42cc)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:38.286801 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91481b1-4eaa-48bd-a6fd-de1da1f0c98e None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:38.288450 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 583/1145] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:30:38 2022] GET /identity/v3/projects/742d51a6036b48198fb1954a1e1f42cc => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:38.294726 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812f7521-0145-4f4e-860c-f67ff5ea6d91 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:38.399454 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812f7521-0145-4f4e-860c-f67ff5ea6d91 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:38.436507 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af1c97-3b99-4376-a01b-b0c27056edc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:38.437135 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af1c97-3b99-4376-a01b-b0c27056edc2 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:38.437413 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af1c97-3b99-4376-a01b-b0c27056edc2 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:38.437682 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af1c97-3b99-4376-a01b-b0c27056edc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:38.438391 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af1c97-3b99-4376-a01b-b0c27056edc2 None admin] RBAC: Adding query filter params (role.id=882f35bd847a406eb2222aeafc517a46, scope.project.id=742d51a6036b48198fb1954a1e1f42cc, user.id=b2f56c180d564d26a95d849fbebc9803) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:30:38.438808 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af1c97-3b99-4376-a01b-b0c27056edc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:38.440241 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af1c97-3b99-4376-a01b-b0c27056edc2 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:38.445739 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 563/1146] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:30:38 2022] GET /identity/v3/role_assignments?user.id=b2f56c180d564d26a95d849fbebc9803&scope.project.id=742d51a6036b48198fb1954a1e1f42cc&role.id=882f35bd847a406eb2222aeafc517a46 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:40.368372 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b6b11-3792-422f-8f99-7f050ded07d6 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:40.368941 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b6b11-3792-422f-8f99-7f050ded07d6 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:40.369047 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b6b11-3792-422f-8f99-7f050ded07d6 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:40.370336 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 584/1147] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:30:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:30:40.375750 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b67a9-63fe-4885-a930-e266836b12b6 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:40.376247 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b67a9-63fe-4885-a930-e266836b12b6 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:40.376459 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b67a9-63fe-4885-a930-e266836b12b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:40.395542 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-d18b67a9-63fe-4885-a930-e266836b12b6 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:30:40.396741 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-d18b67a9-63fe-4885-a930-e266836b12b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:30:40.411659 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:40.465354 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 564/1148] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:30:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:30:40.733016 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b6b11-3792-422f-8f99-7f050ded07d6 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:40.843369 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b6b11-3792-422f-8f99-7f050ded07d6 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:40.882530 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039f0f99-20a6-45d3-b33a-c08af1d8c240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:40.883166 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f0f99-20a6-45d3-b33a-c08af1d8c240 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:40.883442 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f0f99-20a6-45d3-b33a-c08af1d8c240 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:40.883717 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f0f99-20a6-45d3-b33a-c08af1d8c240 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:40.896098 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039f0f99-20a6-45d3-b33a-c08af1d8c240 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:30:40.900991 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-039f0f99-20a6-45d3-b33a-c08af1d8c240 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:30:40.900991 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:30:40.900991 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:30:40.900991 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:30:40.900991 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:30:40.900991 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:30:40.900991 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:30:40.900991 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:30:40.900991 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:30:40.900991 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:30:40.900991 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:30:40.900991 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:30:40.900991 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:30:40.900991 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:30:40.900991 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:30:40.900991 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:30:40.901988 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:30:40.901988 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:30:40.901988 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:30:40.901988 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:30:40.901988 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:30:40.901988 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:30:40.901988 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:30:40.901988 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:30:40.901988 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:30:40.901988 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:30:40.901988 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:30:40.901988 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:30:40.901988 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:30:40.901988 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:30:40.901988 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:30:40.901988 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:30:40.903094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:30:40.903094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:30:40.903094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:30:40.903094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:30:40.903094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:30:40.903094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:30:40.903094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:30:40.903094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:30:40.903094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:30:40.903094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:30:40.903094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:30:40.903094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:30:40.903094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 585/1149] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Aug 29 22:30:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:30:40.908556 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18b67a9-63fe-4885-a930-e266836b12b6 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:41.035955 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18b67a9-63fe-4885-a930-e266836b12b6 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:41.073219 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85e4d44-4fe7-4a17-9ea0-f3c15292f094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:41.073811 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e4d44-4fe7-4a17-9ea0-f3c15292f094 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:41.074238 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e4d44-4fe7-4a17-9ea0-f3c15292f094 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:41.074395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e4d44-4fe7-4a17-9ea0-f3c15292f094 None admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:41.075048 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85e4d44-4fe7-4a17-9ea0-f3c15292f094 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:30:41.075350 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85e4d44-4fe7-4a17-9ea0-f3c15292f094 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:41.076743 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85e4d44-4fe7-4a17-9ea0-f3c15292f094 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:41.086326 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 565/1150] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:30:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:41.092215 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039f0f99-20a6-45d3-b33a-c08af1d8c240 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:41.122849 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:41.198735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039f0f99-20a6-45d3-b33a-c08af1d8c240 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:41.234165 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ad5e3b-9343-48d8-baad-ed1d9c2af011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:41.234808 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad5e3b-9343-48d8-baad-ed1d9c2af011 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:41.235201 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad5e3b-9343-48d8-baad-ed1d9c2af011 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:41.235367 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad5e3b-9343-48d8-baad-ed1d9c2af011 None admin] PATH_INFO: `/v3/users/b2f56c180d564d26a95d849fbebc9803` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:41.237063 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ad5e3b-9343-48d8-baad-ed1d9c2af011 None admin] RBAC: Authorizing `identity:get_user(user_id=b2f56c180d564d26a95d849fbebc9803)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:41.238492 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ad5e3b-9343-48d8-baad-ed1d9c2af011 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:41.240104 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 586/1151] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:30:41 2022] GET /identity/v3/users/b2f56c180d564d26a95d849fbebc9803 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:41.245990 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85e4d44-4fe7-4a17-9ea0-f3c15292f094 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:41.260065 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:41.354481 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85e4d44-4fe7-4a17-9ea0-f3c15292f094 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:41.398030 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217884d5-f718-4124-8645-476001776310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:41.398641 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217884d5-f718-4124-8645-476001776310 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:41.399161 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217884d5-f718-4124-8645-476001776310 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:41.399441 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217884d5-f718-4124-8645-476001776310 None admin] PATH_INFO: `/v3/projects/742d51a6036b48198fb1954a1e1f42cc` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:41.400555 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217884d5-f718-4124-8645-476001776310 None admin] RBAC: Authorizing `identity:get_project(project_id=742d51a6036b48198fb1954a1e1f42cc)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:41.402016 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217884d5-f718-4124-8645-476001776310 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:41.403649 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 566/1152] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:30:41 2022] GET /identity/v3/projects/742d51a6036b48198fb1954a1e1f42cc => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:41.410615 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ad5e3b-9343-48d8-baad-ed1d9c2af011 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:41.511124 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:41.517089 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ad5e3b-9343-48d8-baad-ed1d9c2af011 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:41.552658 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fdb7b9-390b-49b8-9e18-1d373c4b95db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:41.553356 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fdb7b9-390b-49b8-9e18-1d373c4b95db None admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:41.553684 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fdb7b9-390b-49b8-9e18-1d373c4b95db None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:41.553975 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fdb7b9-390b-49b8-9e18-1d373c4b95db None admin] PATH_INFO: `/v3/projects/742d51a6036b48198fb1954a1e1f42cc/users/b2f56c180d564d26a95d849fbebc9803/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:41.556501 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fdb7b9-390b-49b8-9e18-1d373c4b95db None admin] RBAC: Authorizing `identity:create_grant(project_id=742d51a6036b48198fb1954a1e1f42cc, user_id=b2f56c180d564d26a95d849fbebc9803, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:41.557972 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fdb7b9-390b-49b8-9e18-1d373c4b95db None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:41.653680 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 587/1153] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Aug 29 22:30:41 2022] PUT /identity/v3/projects/742d51a6036b48198fb1954a1e1f42cc/users/b2f56c180d564d26a95d849fbebc9803/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:30:43.610896 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f77f3e-df5a-4f1b-a83d-8f9810d1bfca None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:43.612013 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f77f3e-df5a-4f1b-a83d-8f9810d1bfca None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:43.612147 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f77f3e-df5a-4f1b-a83d-8f9810d1bfca None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:43.613664 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 567/1154] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:30:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:30:43.619525 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01e161-5563-4f86-a011-a18bccfee3ef None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:43.620046 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01e161-5563-4f86-a011-a18bccfee3ef None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:43.620046 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01e161-5563-4f86-a011-a18bccfee3ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:43.639485 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-3d01e161-5563-4f86-a011-a18bccfee3ef None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:30:43.640677 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-3d01e161-5563-4f86-a011-a18bccfee3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:30:43.711316 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 588/1155] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:30:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:30:43.985581 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f77f3e-df5a-4f1b-a83d-8f9810d1bfca None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:44.085102 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:44.110004 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f77f3e-df5a-4f1b-a83d-8f9810d1bfca None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:44.149365 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f762bcf-a2ea-4580-89b2-9363fb6e808e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:44.150019 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f762bcf-a2ea-4580-89b2-9363fb6e808e None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:44.150330 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f762bcf-a2ea-4580-89b2-9363fb6e808e None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:44.150577 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f762bcf-a2ea-4580-89b2-9363fb6e808e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:44.167928 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f762bcf-a2ea-4580-89b2-9363fb6e808e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:30:44.173227 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-0f762bcf-a2ea-4580-89b2-9363fb6e808e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:30:44.173227 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:30:44.173227 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:30:44.173227 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:30:44.173227 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:30:44.173227 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:30:44.173227 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:30:44.173227 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:30:44.173227 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:30:44.173227 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:30:44.173227 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:30:44.173227 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:30:44.173227 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:30:44.173227 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:30:44.173227 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:30:44.173227 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:30:44.175558 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:30:44.175558 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:30:44.175558 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:30:44.175558 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:30:44.175558 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:30:44.175558 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:30:44.175558 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:30:44.175558 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:30:44.175558 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:30:44.175558 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:30:44.175558 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:30:44.175558 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:30:44.175558 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:30:44.175558 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:30:44.175558 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:30:44.175558 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:30:44.177265 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:30:44.177265 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:30:44.177265 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:30:44.177265 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:30:44.177265 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:30:44.177265 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:30:44.177265 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:30:44.177265 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:30:44.177265 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:30:44.177265 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:30:44.177265 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:30:44.177265 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:30:44.177265 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 568/1156] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Aug 29 22:30:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:30:44.181869 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01e161-5563-4f86-a011-a18bccfee3ef None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:44.289667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:44.306588 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01e161-5563-4f86-a011-a18bccfee3ef None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:44.345418 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9003a57-84d7-4b0a-8abb-005063908aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:44.346091 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9003a57-84d7-4b0a-8abb-005063908aef None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:44.346371 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9003a57-84d7-4b0a-8abb-005063908aef None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:44.346644 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9003a57-84d7-4b0a-8abb-005063908aef None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:44.347322 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9003a57-84d7-4b0a-8abb-005063908aef None admin] RBAC: Adding query filter params (name=admin) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:30:44.347668 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9003a57-84d7-4b0a-8abb-005063908aef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:44.349156 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9003a57-84d7-4b0a-8abb-005063908aef None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:44.358271 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 589/1157] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:30:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:44.367665 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f762bcf-a2ea-4580-89b2-9363fb6e808e None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:44.481784 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f762bcf-a2ea-4580-89b2-9363fb6e808e None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:44.518387 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90080594-3075-4790-890e-2356118e4e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:44.519012 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90080594-3075-4790-890e-2356118e4e5b None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:44.519283 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90080594-3075-4790-890e-2356118e4e5b None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:44.519739 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90080594-3075-4790-890e-2356118e4e5b None admin] PATH_INFO: `/v3/users/b2f56c180d564d26a95d849fbebc9803` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:44.520902 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90080594-3075-4790-890e-2356118e4e5b None admin] RBAC: Authorizing `identity:get_user(user_id=b2f56c180d564d26a95d849fbebc9803)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:44.522330 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90080594-3075-4790-890e-2356118e4e5b None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:44.523903 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 569/1158] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:30:44 2022] GET /identity/v3/users/b2f56c180d564d26a95d849fbebc9803 => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:44.530000 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9003a57-84d7-4b0a-8abb-005063908aef None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:44.652036 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9003a57-84d7-4b0a-8abb-005063908aef None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:44.689624 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214430f-0ede-4113-a774-b52a3e12c739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:44.690675 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214430f-0ede-4113-a774-b52a3e12c739 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:44.690675 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214430f-0ede-4113-a774-b52a3e12c739 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:44.690893 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214430f-0ede-4113-a774-b52a3e12c739 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:44.709367 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:44.763669 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3214430f-0ede-4113-a774-b52a3e12c739 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:44.765169 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3214430f-0ede-4113-a774-b52a3e12c739 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:44.870776 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 590/1159] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:30:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 22:30:44.876961 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90080594-3075-4790-890e-2356118e4e5b None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:44.885967 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:44.982354 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90080594-3075-4790-890e-2356118e4e5b None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:45.021417 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a995171-2b64-4a95-85cb-21ca8281d2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:45.022050 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a995171-2b64-4a95-85cb-21ca8281d2b4 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:45.022332 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a995171-2b64-4a95-85cb-21ca8281d2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:45.022604 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a995171-2b64-4a95-85cb-21ca8281d2b4 None admin] PATH_INFO: `/v3/projects/742d51a6036b48198fb1954a1e1f42cc` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:45.023719 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a995171-2b64-4a95-85cb-21ca8281d2b4 None admin] RBAC: Authorizing `identity:get_project(project_id=742d51a6036b48198fb1954a1e1f42cc)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:45.025295 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a995171-2b64-4a95-85cb-21ca8281d2b4 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:45.026919 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 570/1160] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:30:44 2022] GET /identity/v3/projects/742d51a6036b48198fb1954a1e1f42cc => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:45.033816 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214430f-0ede-4113-a774-b52a3e12c739 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:45.086112 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:45.138111 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214430f-0ede-4113-a774-b52a3e12c739 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:45.174663 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc7ba18-7096-4be2-8907-20ca6befdee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:45.175275 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc7ba18-7096-4be2-8907-20ca6befdee1 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:45.175558 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc7ba18-7096-4be2-8907-20ca6befdee1 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:45.175833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc7ba18-7096-4be2-8907-20ca6befdee1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:45.176840 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc7ba18-7096-4be2-8907-20ca6befdee1 None admin] RBAC: Adding query filter params (role.id=882f35bd847a406eb2222aeafc517a46, scope.project.id=742d51a6036b48198fb1954a1e1f42cc, user.id=b2f56c180d564d26a95d849fbebc9803) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:30:45.177207 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc7ba18-7096-4be2-8907-20ca6befdee1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:45.178658 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc7ba18-7096-4be2-8907-20ca6befdee1 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:45.184790 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 591/1161] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:30:45 2022] GET /identity/v3/role_assignments?user.id=b2f56c180d564d26a95d849fbebc9803&scope.project.id=742d51a6036b48198fb1954a1e1f42cc&role.id=882f35bd847a406eb2222aeafc517a46 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:47.141338 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e55f9-ba53-44d7-936a-e3a7b0345c75 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:47.141868 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e55f9-ba53-44d7-936a-e3a7b0345c75 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:47.141868 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e55f9-ba53-44d7-936a-e3a7b0345c75 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:47.143981 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 571/1162] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:30:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:30:47.149408 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e46a427-8b0c-4955-9568-6c9915ca7da4 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:47.149814 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e46a427-8b0c-4955-9568-6c9915ca7da4 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:47.149929 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e46a427-8b0c-4955-9568-6c9915ca7da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:47.168424 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-7e46a427-8b0c-4955-9568-6c9915ca7da4 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:30:47.169653 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-7e46a427-8b0c-4955-9568-6c9915ca7da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:30:47.235539 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 592/1163] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:30:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:30:47.512907 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424e55f9-ba53-44d7-936a-e3a7b0345c75 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:47.593572 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:47.619919 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424e55f9-ba53-44d7-936a-e3a7b0345c75 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:47.654386 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61dc904-1702-4707-af94-1dee7cba7fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:47.655004 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61dc904-1702-4707-af94-1dee7cba7fc2 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:47.655286 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61dc904-1702-4707-af94-1dee7cba7fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:47.655871 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61dc904-1702-4707-af94-1dee7cba7fc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:47.725461 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61dc904-1702-4707-af94-1dee7cba7fc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:47.726904 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61dc904-1702-4707-af94-1dee7cba7fc2 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:47.799694 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:47.824853 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 572/1164] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:30:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 22:30:47.830822 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e46a427-8b0c-4955-9568-6c9915ca7da4 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:47.873545 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:47.933957 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e46a427-8b0c-4955-9568-6c9915ca7da4 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:47.967624 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc49e4c-58a0-419d-a565-96c385748e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:47.968244 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc49e4c-58a0-419d-a565-96c385748e0b None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:47.968525 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc49e4c-58a0-419d-a565-96c385748e0b None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:47.968825 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc49e4c-58a0-419d-a565-96c385748e0b None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:47.976089 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc49e4c-58a0-419d-a565-96c385748e0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:47.977476 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc49e4c-58a0-419d-a565-96c385748e0b None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:47.990537 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-3dc49e4c-58a0-419d-a565-96c385748e0b None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 29 22:30:47.990537 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:30:47.990537 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 29 22:30:47.990537 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 29 22:30:47.990537 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:30:47.990537 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:30:47.990537 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:30:47.990537 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:30:47.990537 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 29 22:30:47.990537 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:30:47.990537 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:30:47.990537 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:30:47.990537 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:30:47.990537 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:30:47.990537 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:30:47.990537 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:30:47.990537 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:30:47.991824 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:30:47.991824 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:30:47.991824 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:30:47.991824 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:30:47.991824 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:30:47.991824 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:30:47.991824 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:30:47.991824 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:30:47.991824 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:30:47.991824 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:30:47.991824 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:30:47.991824 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:30:47.991824 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:30:47.991824 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:30:47.991824 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:30:47.991824 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:30:47.992898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:30:47.992898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:30:47.992898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:30:47.992898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:30:47.992898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:30:47.992898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:30:47.992898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:30:47.992898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:30:47.992898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:30:47.992898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:30:47.992898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 29 22:30:47.992898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:30:47.992898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 29 22:30:47.992898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:30:47.992898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 593/1165] 10.222.0.52 () {56 vars in 1117 bytes} [Mon Aug 29 22:30:47 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:30:47.997865 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61dc904-1702-4707-af94-1dee7cba7fc2 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:48.098059 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61dc904-1702-4707-af94-1dee7cba7fc2 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:48.134646 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c26df80-d101-4391-a14b-ebc5c0fea3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:48.135249 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26df80-d101-4391-a14b-ebc5c0fea3e5 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:48.135527 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26df80-d101-4391-a14b-ebc5c0fea3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:48.135800 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26df80-d101-4391-a14b-ebc5c0fea3e5 None admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:48.136450 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c26df80-d101-4391-a14b-ebc5c0fea3e5 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:30:48.136843 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c26df80-d101-4391-a14b-ebc5c0fea3e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:48.138155 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c26df80-d101-4391-a14b-ebc5c0fea3e5 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:48.146690 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 573/1166] 10.222.0.52 () {56 vars in 1115 bytes} [Mon Aug 29 22:30:47 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:50.100379 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a71ac-0233-4f3b-96c2-b070ed395c79 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:50.100975 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a71ac-0233-4f3b-96c2-b070ed395c79 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:50.100975 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a71ac-0233-4f3b-96c2-b070ed395c79 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:50.102286 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 594/1167] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:30:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:30:50.107405 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0055302-d513-4263-805e-40f9b12a4feb None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:50.107836 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0055302-d513-4263-805e-40f9b12a4feb None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:50.108016 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0055302-d513-4263-805e-40f9b12a4feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:50.129832 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-a0055302-d513-4263-805e-40f9b12a4feb None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:30:50.131022 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-a0055302-d513-4263-805e-40f9b12a4feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:30:50.165755 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:50.204283 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 574/1168] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:30:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:30:50.471126 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a71ac-0233-4f3b-96c2-b070ed395c79 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:50.575867 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a71ac-0233-4f3b-96c2-b070ed395c79 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:50.577490 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:50.610380 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4485ca-ac51-48b4-9479-1e5fadac54f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:50.610976 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4485ca-ac51-48b4-9479-1e5fadac54f9 None admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:50.611304 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4485ca-ac51-48b4-9479-1e5fadac54f9 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:50.611579 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4485ca-ac51-48b4-9479-1e5fadac54f9 None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:50.612343 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4485ca-ac51-48b4-9479-1e5fadac54f9 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:50.613736 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4485ca-ac51-48b4-9479-1e5fadac54f9 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:50.693410 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 595/1169] 10.222.0.52 () {60 vars in 1138 bytes} [Mon Aug 29 22:30:50 2022] POST /identity/v3/domains => generated 267 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:30:52.695306 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba2649-5474-41c6-9cb0-5b76d76f2695 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:52.695889 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba2649-5474-41c6-9cb0-5b76d76f2695 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:52.695889 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba2649-5474-41c6-9cb0-5b76d76f2695 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:52.697249 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 575/1170] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:30:52.702935 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aeced7-304c-4151-ba67-792d5827aed3 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:52.703439 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aeced7-304c-4151-ba67-792d5827aed3 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:52.703537 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aeced7-304c-4151-ba67-792d5827aed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:52.722762 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-c3aeced7-304c-4151-ba67-792d5827aed3 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:30:52.723969 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-c3aeced7-304c-4151-ba67-792d5827aed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:30:52.805251 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 596/1171] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:30:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:30:53.080018 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeba2649-5474-41c6-9cb0-5b76d76f2695 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:53.183737 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeba2649-5474-41c6-9cb0-5b76d76f2695 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:53.217095 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21226127-0395-4f74-9ccc-fb7848bf6c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:53.217717 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21226127-0395-4f74-9ccc-fb7848bf6c51 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:53.217993 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21226127-0395-4f74-9ccc-fb7848bf6c51 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:53.218265 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21226127-0395-4f74-9ccc-fb7848bf6c51 None admin] PATH_INFO: `/v3/domains/3a51a2dadff747aabea074069d78d065` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:53.229084 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21226127-0395-4f74-9ccc-fb7848bf6c51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=3a51a2dadff747aabea074069d78d065)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:53.230516 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21226127-0395-4f74-9ccc-fb7848bf6c51 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:53.232097 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 576/1172] 10.222.0.52 () {56 vars in 1183 bytes} [Mon Aug 29 22:30:53 2022] GET /identity/v3/domains/3a51a2dadff747aabea074069d78d065 => generated 267 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:53.238340 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aeced7-304c-4151-ba67-792d5827aed3 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:53.328143 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:53.346780 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aeced7-304c-4151-ba67-792d5827aed3 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:53.370500 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800738da-8f6e-4a68-93d0-c492315374df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:53.371217 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800738da-8f6e-4a68-93d0-c492315374df None admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:53.371835 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800738da-8f6e-4a68-93d0-c492315374df None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:53.372293 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800738da-8f6e-4a68-93d0-c492315374df None admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:53.373367 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800738da-8f6e-4a68-93d0-c492315374df None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:53.374834 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800738da-8f6e-4a68-93d0-c492315374df None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:53.487850 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 597/1173] 10.222.0.52 () {60 vars in 1141 bytes} [Mon Aug 29 22:30:53 2022] POST /identity/v3/projects => generated 361 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:30:55.448828 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc4753-cc7e-438a-8d27-0d87d362c754 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:55.449360 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc4753-cc7e-438a-8d27-0d87d362c754 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:55.449360 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc4753-cc7e-438a-8d27-0d87d362c754 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:55.450693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 577/1174] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:30:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:30:55.455782 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccbcdbf-aa44-4f63-b8db-2fa10817a8f6 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:55.456203 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccbcdbf-aa44-4f63-b8db-2fa10817a8f6 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:55.456305 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccbcdbf-aa44-4f63-b8db-2fa10817a8f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:55.474308 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-1ccbcdbf-aa44-4f63-b8db-2fa10817a8f6 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:30:55.475490 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-1ccbcdbf-aa44-4f63-b8db-2fa10817a8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:30:55.543691 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 598/1175] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:30:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:30:55.814860 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc4753-cc7e-438a-8d27-0d87d362c754 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:55.846155 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:55.923139 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc4753-cc7e-438a-8d27-0d87d362c754 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:55.956996 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4086a0-2e75-47ac-8a44-684d72bfb18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:55.957611 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4086a0-2e75-47ac-8a44-684d72bfb18e None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:55.957892 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4086a0-2e75-47ac-8a44-684d72bfb18e None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:55.958166 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4086a0-2e75-47ac-8a44-684d72bfb18e None admin] PATH_INFO: `/v3/domains/3a51a2dadff747aabea074069d78d065` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:55.959294 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4086a0-2e75-47ac-8a44-684d72bfb18e None admin] RBAC: Authorizing `identity:get_domain(domain_id=3a51a2dadff747aabea074069d78d065)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:55.960808 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4086a0-2e75-47ac-8a44-684d72bfb18e None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:55.965936 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 578/1176] 10.222.0.52 () {56 vars in 1183 bytes} [Mon Aug 29 22:30:55 2022] GET /identity/v3/domains/3a51a2dadff747aabea074069d78d065 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:55.968277 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccbcdbf-aa44-4f63-b8db-2fa10817a8f6 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:56.044092 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:56.069870 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccbcdbf-aa44-4f63-b8db-2fa10817a8f6 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:56.104037 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b485f52-f857-414a-89f6-bd6de66bbe8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:56.104631 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b485f52-f857-414a-89f6-bd6de66bbe8c None admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:56.104937 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b485f52-f857-414a-89f6-bd6de66bbe8c None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:56.105213 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b485f52-f857-414a-89f6-bd6de66bbe8c None admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:56.105850 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b485f52-f857-414a-89f6-bd6de66bbe8c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:56.107412 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b485f52-f857-414a-89f6-bd6de66bbe8c None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:56.259077 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 599/1177] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Aug 29 22:30:55 2022] POST /identity/v3/users => generated 311 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:30:58.191648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7288d6-50e7-467e-b97e-1ddb6dcbf096 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:58.192179 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7288d6-50e7-467e-b97e-1ddb6dcbf096 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:58.192179 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7288d6-50e7-467e-b97e-1ddb6dcbf096 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:58.193532 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 579/1178] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:30:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:30:58.198881 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc314391-ae5a-4b1b-a5eb-d9e614547ce6 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:58.199287 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc314391-ae5a-4b1b-a5eb-d9e614547ce6 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:58.199436 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc314391-ae5a-4b1b-a5eb-d9e614547ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:58.220065 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-fc314391-ae5a-4b1b-a5eb-d9e614547ce6 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:30:58.221383 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-fc314391-ae5a-4b1b-a5eb-d9e614547ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:30:58.290541 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 600/1179] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:30:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:30:58.562866 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7288d6-50e7-467e-b97e-1ddb6dcbf096 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:58.657838 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:58.664477 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7288d6-50e7-467e-b97e-1ddb6dcbf096 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:58.698936 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117e7a28-452e-4c86-bb4c-da8ec3e52457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:58.699554 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e7a28-452e-4c86-bb4c-da8ec3e52457 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:58.699838 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e7a28-452e-4c86-bb4c-da8ec3e52457 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:58.700117 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e7a28-452e-4c86-bb4c-da8ec3e52457 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:58.710130 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117e7a28-452e-4c86-bb4c-da8ec3e52457 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:30:58.714912 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-117e7a28-452e-4c86-bb4c-da8ec3e52457 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:30:58.714912 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:30:58.714912 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:30:58.714912 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:30:58.714912 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:30:58.714912 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:30:58.714912 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:30:58.714912 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:30:58.714912 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:30:58.714912 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:30:58.714912 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:30:58.714912 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:30:58.714912 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:30:58.714912 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:30:58.714912 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:30:58.714912 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:30:58.716247 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:30:58.716247 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:30:58.716247 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:30:58.716247 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:30:58.716247 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:30:58.716247 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:30:58.716247 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:30:58.716247 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:30:58.716247 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:30:58.716247 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:30:58.716247 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:30:58.716247 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:30:58.716247 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:30:58.716247 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:30:58.716247 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:30:58.716247 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:30:58.717248 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:30:58.717248 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:30:58.717248 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:30:58.717248 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:30:58.717248 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:30:58.717248 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:30:58.717248 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:30:58.717248 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:30:58.717248 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:30:58.717248 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:30:58.717248 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:30:58.717248 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:30:58.717248 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 580/1180] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Aug 29 22:30:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:30:58.722334 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc314391-ae5a-4b1b-a5eb-d9e614547ce6 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:58.781527 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:58.822801 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc314391-ae5a-4b1b-a5eb-d9e614547ce6 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:58.857744 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd581094-466f-432d-a3b2-1c02bb1c6b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:58.858348 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd581094-466f-432d-a3b2-1c02bb1c6b40 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:58.858625 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd581094-466f-432d-a3b2-1c02bb1c6b40 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:58.858894 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd581094-466f-432d-a3b2-1c02bb1c6b40 None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:58.859545 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd581094-466f-432d-a3b2-1c02bb1c6b40 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:30:58.859841 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd581094-466f-432d-a3b2-1c02bb1c6b40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:58.861821 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd581094-466f-432d-a3b2-1c02bb1c6b40 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:58.869959 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 601/1181] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:30:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:58.875930 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117e7a28-452e-4c86-bb4c-da8ec3e52457 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:58.976130 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117e7a28-452e-4c86-bb4c-da8ec3e52457 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:59.010494 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dabacf-a7cb-46cf-b017-59d828158ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:59.011154 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dabacf-a7cb-46cf-b017-59d828158ecc None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:59.011432 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dabacf-a7cb-46cf-b017-59d828158ecc None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:59.011704 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dabacf-a7cb-46cf-b017-59d828158ecc None admin] PATH_INFO: `/v3/users/0fb8591f8ddd4dd99925fd21f65d18ec` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:59.025121 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dabacf-a7cb-46cf-b017-59d828158ecc None admin] RBAC: Authorizing `identity:get_user(user_id=0fb8591f8ddd4dd99925fd21f65d18ec)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:59.026552 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dabacf-a7cb-46cf-b017-59d828158ecc None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:59.028139 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 581/1182] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:30:58 2022] GET /identity/v3/users/0fb8591f8ddd4dd99925fd21f65d18ec => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:59.034716 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd581094-466f-432d-a3b2-1c02bb1c6b40 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:59.136766 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd581094-466f-432d-a3b2-1c02bb1c6b40 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:59.159999 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:59.171717 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40336764-8c4d-4ddc-9e2a-9e50ca94e33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:59.172358 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40336764-8c4d-4ddc-9e2a-9e50ca94e33b None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:59.172637 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40336764-8c4d-4ddc-9e2a-9e50ca94e33b None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:59.172936 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40336764-8c4d-4ddc-9e2a-9e50ca94e33b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:59.258471 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40336764-8c4d-4ddc-9e2a-9e50ca94e33b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:59.259895 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40336764-8c4d-4ddc-9e2a-9e50ca94e33b None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:59.358898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 602/1183] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:30:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 22:30:59.365402 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dabacf-a7cb-46cf-b017-59d828158ecc None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:59.381272 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:59.465594 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dabacf-a7cb-46cf-b017-59d828158ecc None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:59.499571 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8687bda-2892-48d2-9ad3-c7dbf99af6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:59.500192 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8687bda-2892-48d2-9ad3-c7dbf99af6ed None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:59.500469 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8687bda-2892-48d2-9ad3-c7dbf99af6ed None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:59.500764 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8687bda-2892-48d2-9ad3-c7dbf99af6ed None admin] PATH_INFO: `/v3/projects/30a67ca006fa491ca0f601d813eb2b59` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:59.502402 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8687bda-2892-48d2-9ad3-c7dbf99af6ed None admin] RBAC: Authorizing `identity:get_project(project_id=30a67ca006fa491ca0f601d813eb2b59)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:59.504001 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8687bda-2892-48d2-9ad3-c7dbf99af6ed None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:59.505645 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 582/1184] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:30:59 2022] GET /identity/v3/projects/30a67ca006fa491ca0f601d813eb2b59 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:30:59.511772 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40336764-8c4d-4ddc-9e2a-9e50ca94e33b None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:30:59.536163 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:30:59.611340 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40336764-8c4d-4ddc-9e2a-9e50ca94e33b None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:30:59.645296 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcb8d7c-2bdc-47cc-9a1b-03136254eb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:30:59.645893 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb8d7c-2bdc-47cc-9a1b-03136254eb82 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:30:59.646166 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb8d7c-2bdc-47cc-9a1b-03136254eb82 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:30:59.646435 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb8d7c-2bdc-47cc-9a1b-03136254eb82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:30:59.647149 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcb8d7c-2bdc-47cc-9a1b-03136254eb82 None admin] RBAC: Adding query filter params (role.id=882f35bd847a406eb2222aeafc517a46, scope.project.id=30a67ca006fa491ca0f601d813eb2b59, user.id=0fb8591f8ddd4dd99925fd21f65d18ec) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:30:59.647452 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcb8d7c-2bdc-47cc-9a1b-03136254eb82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:30:59.648959 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcb8d7c-2bdc-47cc-9a1b-03136254eb82 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:30:59.654175 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 603/1185] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:30:59 2022] GET /identity/v3/role_assignments?user.id=0fb8591f8ddd4dd99925fd21f65d18ec&scope.project.id=30a67ca006fa491ca0f601d813eb2b59&role.id=882f35bd847a406eb2222aeafc517a46 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:31:01.571339 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce654c4-9c09-464b-9f73-603cf0ecf868 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:31:01.571868 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce654c4-9c09-464b-9f73-603cf0ecf868 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:31:01.571868 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce654c4-9c09-464b-9f73-603cf0ecf868 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:31:01.573312 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 583/1186] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:31:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:31:01.578517 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd90dc-97ca-4070-9daa-0cacb0ce0471 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:31:01.579035 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd90dc-97ca-4070-9daa-0cacb0ce0471 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:31:01.579141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd90dc-97ca-4070-9daa-0cacb0ce0471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:31:01.597629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-23bd90dc-97ca-4070-9daa-0cacb0ce0471 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:31:01.598811 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-23bd90dc-97ca-4070-9daa-0cacb0ce0471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:31:01.664827 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 604/1187] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:31:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:31:01.934139 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce654c4-9c09-464b-9f73-603cf0ecf868 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:31:02.020815 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:31:02.040013 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce654c4-9c09-464b-9f73-603cf0ecf868 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:31:02.074980 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a73bb06-f8d1-4b51-967e-f761f3aae63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:31:02.075602 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73bb06-f8d1-4b51-967e-f761f3aae63d None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:31:02.075885 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73bb06-f8d1-4b51-967e-f761f3aae63d None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:31:02.076151 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73bb06-f8d1-4b51-967e-f761f3aae63d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:31:02.086746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a73bb06-f8d1-4b51-967e-f761f3aae63d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:31:02.091525 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-0a73bb06-f8d1-4b51-967e-f761f3aae63d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:31:02.091525 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:31:02.091525 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:31:02.091525 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:31:02.091525 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:31:02.091525 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:31:02.091525 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:31:02.091525 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:31:02.091525 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:31:02.091525 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:31:02.091525 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:31:02.091525 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:31:02.091525 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:31:02.091525 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:31:02.091525 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:31:02.091525 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:31:02.092650 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:31:02.092650 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:31:02.092650 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:31:02.092650 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:31:02.092650 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:31:02.092650 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:31:02.092650 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:31:02.092650 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:31:02.092650 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:31:02.092650 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:31:02.092650 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:31:02.092650 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:31:02.092650 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:31:02.092650 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:31:02.092650 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:31:02.092650 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:31:02.093974 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:31:02.093974 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:31:02.093974 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:31:02.093974 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:31:02.093974 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:31:02.093974 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:31:02.093974 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:31:02.093974 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:31:02.093974 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:31:02.093974 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:31:02.093974 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:31:02.093974 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:31:02.093974 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 584/1188] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Aug 29 22:31:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:31:02.099537 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bd90dc-97ca-4070-9daa-0cacb0ce0471 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:31:02.117307 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:31:02.200127 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bd90dc-97ca-4070-9daa-0cacb0ce0471 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:31:02.236591 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0451c-827e-4205-82ba-b96659aff8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:31:02.237210 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0451c-827e-4205-82ba-b96659aff8d2 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:31:02.237484 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0451c-827e-4205-82ba-b96659aff8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:31:02.237760 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0451c-827e-4205-82ba-b96659aff8d2 None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:31:02.238405 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c0451c-827e-4205-82ba-b96659aff8d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:31:02.238715 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c0451c-827e-4205-82ba-b96659aff8d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:31:02.240050 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c0451c-827e-4205-82ba-b96659aff8d2 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:31:02.248003 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 605/1189] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:31:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:31:02.254456 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a73bb06-f8d1-4b51-967e-f761f3aae63d None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:31:02.361402 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a73bb06-f8d1-4b51-967e-f761f3aae63d None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:31:02.394844 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e3cf5-4e38-4946-97a2-0d1b1ef0e3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:31:02.395461 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e3cf5-4e38-4946-97a2-0d1b1ef0e3bd None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:31:02.395734 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e3cf5-4e38-4946-97a2-0d1b1ef0e3bd None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:31:02.396004 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e3cf5-4e38-4946-97a2-0d1b1ef0e3bd None admin] PATH_INFO: `/v3/users/0fb8591f8ddd4dd99925fd21f65d18ec` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:31:02.397130 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e3cf5-4e38-4946-97a2-0d1b1ef0e3bd None admin] RBAC: Authorizing `identity:get_user(user_id=0fb8591f8ddd4dd99925fd21f65d18ec)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:31:02.398552 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e3cf5-4e38-4946-97a2-0d1b1ef0e3bd None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:31:02.400123 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 585/1190] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:31:02 2022] GET /identity/v3/users/0fb8591f8ddd4dd99925fd21f65d18ec => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:31:02.406154 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0451c-827e-4205-82ba-b96659aff8d2 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:31:02.486507 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:31:02.508554 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0451c-827e-4205-82ba-b96659aff8d2 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:31:02.544883 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6958ad-4b24-4068-8116-68aa866e9748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:31:02.545527 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6958ad-4b24-4068-8116-68aa866e9748 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:31:02.545800 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6958ad-4b24-4068-8116-68aa866e9748 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:31:02.546077 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6958ad-4b24-4068-8116-68aa866e9748 None admin] PATH_INFO: `/v3/projects/30a67ca006fa491ca0f601d813eb2b59` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:31:02.547195 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6958ad-4b24-4068-8116-68aa866e9748 None admin] RBAC: Authorizing `identity:get_project(project_id=30a67ca006fa491ca0f601d813eb2b59)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:31:02.549370 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6958ad-4b24-4068-8116-68aa866e9748 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:31:02.551135 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 606/1191] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:31:02 2022] GET /identity/v3/projects/30a67ca006fa491ca0f601d813eb2b59 => generated 361 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:31:02.557498 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e3cf5-4e38-4946-97a2-0d1b1ef0e3bd None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:31:02.570328 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:31:02.657018 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e3cf5-4e38-4946-97a2-0d1b1ef0e3bd None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:31:02.691007 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f3c1b9-17b1-474d-9ef3-a6f59b24a22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:31:02.691651 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f3c1b9-17b1-474d-9ef3-a6f59b24a22b None admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:31:02.691932 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f3c1b9-17b1-474d-9ef3-a6f59b24a22b None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:31:02.692203 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f3c1b9-17b1-474d-9ef3-a6f59b24a22b None admin] PATH_INFO: `/v3/projects/30a67ca006fa491ca0f601d813eb2b59/users/0fb8591f8ddd4dd99925fd21f65d18ec/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:31:02.694106 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f3c1b9-17b1-474d-9ef3-a6f59b24a22b None admin] RBAC: Authorizing `identity:create_grant(project_id=30a67ca006fa491ca0f601d813eb2b59, user_id=0fb8591f8ddd4dd99925fd21f65d18ec, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:31:02.695532 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f3c1b9-17b1-474d-9ef3-a6f59b24a22b None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:31:02.761168 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 586/1192] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Aug 29 22:31:02 2022] PUT /identity/v3/projects/30a67ca006fa491ca0f601d813eb2b59/users/0fb8591f8ddd4dd99925fd21f65d18ec/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:31:04.666243 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf473a92-90c1-46ca-ad1f-b1f7ac2d928f None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:31:04.666771 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf473a92-90c1-46ca-ad1f-b1f7ac2d928f None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:31:04.666771 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf473a92-90c1-46ca-ad1f-b1f7ac2d928f None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:31:04.668124 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 607/1193] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:31:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:31:04.673599 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e947530-ec65-44d8-bd3b-b711e2cb4a86 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:31:04.674053 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e947530-ec65-44d8-bd3b-b711e2cb4a86 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:31:04.674152 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e947530-ec65-44d8-bd3b-b711e2cb4a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:31:04.692354 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-3e947530-ec65-44d8-bd3b-b711e2cb4a86 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:31:04.693561 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-3e947530-ec65-44d8-bd3b-b711e2cb4a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:31:04.755671 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:31:04.760119 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 587/1194] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:31:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:31:05.029993 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf473a92-90c1-46ca-ad1f-b1f7ac2d928f None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:31:05.133639 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf473a92-90c1-46ca-ad1f-b1f7ac2d928f None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:31:05.166657 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b754b83f-2779-40bc-a725-97c902696610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:31:05.167274 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754b83f-2779-40bc-a725-97c902696610 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:31:05.167555 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754b83f-2779-40bc-a725-97c902696610 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:31:05.167828 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754b83f-2779-40bc-a725-97c902696610 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:31:05.179189 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b754b83f-2779-40bc-a725-97c902696610 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:31:05.184057 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-b754b83f-2779-40bc-a725-97c902696610 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:31:05.184057 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:31:05.184057 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:31:05.184057 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:31:05.184057 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:31:05.184057 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:31:05.184057 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:31:05.184057 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:31:05.184057 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:31:05.184057 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:31:05.184057 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:31:05.184057 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:31:05.184057 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:31:05.184057 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:31:05.184057 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:31:05.184057 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:31:05.185216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:31:05.185216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:31:05.185216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:31:05.185216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:31:05.185216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:31:05.185216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:31:05.185216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:31:05.185216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:31:05.185216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:31:05.185216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:31:05.185216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:31:05.185216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:31:05.185216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:31:05.185216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:31:05.185216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:31:05.185216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:31:05.186241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:31:05.186241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:31:05.186241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:31:05.186241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:31:05.186241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:31:05.186241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:31:05.186241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:31:05.186241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:31:05.186241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:31:05.186241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:31:05.186241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:31:05.186241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:31:05.186241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 608/1195] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Aug 29 22:31:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:31:05.191999 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e947530-ec65-44d8-bd3b-b711e2cb4a86 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:31:05.288817 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e947530-ec65-44d8-bd3b-b711e2cb4a86 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:31:05.323286 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de2bc6-094e-40bb-ae10-60a9b431d260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:31:05.323877 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de2bc6-094e-40bb-ae10-60a9b431d260 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:31:05.324152 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de2bc6-094e-40bb-ae10-60a9b431d260 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:31:05.324419 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de2bc6-094e-40bb-ae10-60a9b431d260 None admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:31:05.325185 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de2bc6-094e-40bb-ae10-60a9b431d260 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:31:05.325427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de2bc6-094e-40bb-ae10-60a9b431d260 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:31:05.326792 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de2bc6-094e-40bb-ae10-60a9b431d260 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:31:05.334916 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 588/1196] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:31:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:31:05.340831 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b754b83f-2779-40bc-a725-97c902696610 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:31:05.350125 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:31:05.441045 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b754b83f-2779-40bc-a725-97c902696610 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:31:05.473537 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9dbd6b-36a0-4d32-affa-8cdd6246fbd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:31:05.474156 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9dbd6b-36a0-4d32-affa-8cdd6246fbd7 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:31:05.474434 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9dbd6b-36a0-4d32-affa-8cdd6246fbd7 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:31:05.474707 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9dbd6b-36a0-4d32-affa-8cdd6246fbd7 None admin] PATH_INFO: `/v3/users/0fb8591f8ddd4dd99925fd21f65d18ec` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:31:05.475804 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9dbd6b-36a0-4d32-affa-8cdd6246fbd7 None admin] RBAC: Authorizing `identity:get_user(user_id=0fb8591f8ddd4dd99925fd21f65d18ec)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:31:05.477264 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9dbd6b-36a0-4d32-affa-8cdd6246fbd7 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:31:05.478837 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 609/1197] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Aug 29 22:31:05 2022] GET /identity/v3/users/0fb8591f8ddd4dd99925fd21f65d18ec => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:31:05.484823 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de2bc6-094e-40bb-ae10-60a9b431d260 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:31:05.546544 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:31:05.584630 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de2bc6-094e-40bb-ae10-60a9b431d260 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:31:05.619223 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98410f53-ad5e-4203-ade8-939e33a031eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:31:05.619826 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98410f53-ad5e-4203-ade8-939e33a031eb None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:31:05.620108 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98410f53-ad5e-4203-ade8-939e33a031eb None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:31:05.620394 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98410f53-ad5e-4203-ade8-939e33a031eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:31:05.687595 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98410f53-ad5e-4203-ade8-939e33a031eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:31:05.689181 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98410f53-ad5e-4203-ade8-939e33a031eb None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:31:05.747422 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:31:05.789090 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 589/1198] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:31:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 22:31:05.795191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9dbd6b-36a0-4d32-affa-8cdd6246fbd7 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:31:05.870844 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:31:05.895236 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9dbd6b-36a0-4d32-affa-8cdd6246fbd7 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:31:05.930157 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad262a40-3878-4a11-8c60-3829f331535e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:31:05.930781 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad262a40-3878-4a11-8c60-3829f331535e None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:31:05.931052 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad262a40-3878-4a11-8c60-3829f331535e None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:31:05.931322 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad262a40-3878-4a11-8c60-3829f331535e None admin] PATH_INFO: `/v3/projects/30a67ca006fa491ca0f601d813eb2b59` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:31:05.932427 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad262a40-3878-4a11-8c60-3829f331535e None admin] RBAC: Authorizing `identity:get_project(project_id=30a67ca006fa491ca0f601d813eb2b59)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:31:05.933880 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad262a40-3878-4a11-8c60-3829f331535e None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:31:05.935500 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 610/1199] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Aug 29 22:31:05 2022] GET /identity/v3/projects/30a67ca006fa491ca0f601d813eb2b59 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:31:05.942277 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98410f53-ad5e-4203-ade8-939e33a031eb None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:31:06.042965 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98410f53-ad5e-4203-ade8-939e33a031eb None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:31:06.076926 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531e3b52-3ea1-4611-80d3-1ac5ebbcae48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:31:06.077517 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e3b52-3ea1-4611-80d3-1ac5ebbcae48 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:31:06.077792 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e3b52-3ea1-4611-80d3-1ac5ebbcae48 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:31:06.078061 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e3b52-3ea1-4611-80d3-1ac5ebbcae48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:31:06.078769 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531e3b52-3ea1-4611-80d3-1ac5ebbcae48 None admin] RBAC: Adding query filter params (role.id=882f35bd847a406eb2222aeafc517a46, scope.project.id=30a67ca006fa491ca0f601d813eb2b59, user.id=0fb8591f8ddd4dd99925fd21f65d18ec) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:31:06.079071 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531e3b52-3ea1-4611-80d3-1ac5ebbcae48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:31:06.081241 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531e3b52-3ea1-4611-80d3-1ac5ebbcae48 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:31:06.086976 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 590/1200] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:31:05 2022] GET /identity/v3/role_assignments?user.id=0fb8591f8ddd4dd99925fd21f65d18ec&scope.project.id=30a67ca006fa491ca0f601d813eb2b59&role.id=882f35bd847a406eb2222aeafc517a46 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:31:15.925598 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=29 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:31:16.084294 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=87 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:23.770798 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51772f-4708-4677-801c-911ea058b984 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:23.770798 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51772f-4708-4677-801c-911ea058b984 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:23.770798 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51772f-4708-4677-801c-911ea058b984 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:23.772197 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 611/1201] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:32:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:32:23.777379 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79909df5-20f0-41de-b217-937ba2219cf3 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:23.777937 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79909df5-20f0-41de-b217-937ba2219cf3 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:23.777937 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79909df5-20f0-41de-b217-937ba2219cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:23.796581 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-79909df5-20f0-41de-b217-937ba2219cf3 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:32:23.797732 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-79909df5-20f0-41de-b217-937ba2219cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:32:23.866538 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 591/1202] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:32:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:32:24.136224 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d51772f-4708-4677-801c-911ea058b984 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:24.240039 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d51772f-4708-4677-801c-911ea058b984 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:24.275130 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a73cb9-39df-43f5-9f3b-d212b93e279a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:24.275745 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a73cb9-39df-43f5-9f3b-d212b93e279a None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:24.276021 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a73cb9-39df-43f5-9f3b-d212b93e279a None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:24.276315 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a73cb9-39df-43f5-9f3b-d212b93e279a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:24.288245 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a73cb9-39df-43f5-9f3b-d212b93e279a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:24.289733 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a73cb9-39df-43f5-9f3b-d212b93e279a None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:24.305597 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-d2a73cb9-39df-43f5-9f3b-d212b93e279a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:32:24.305597 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:32:24.305597 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:32:24.305597 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:32:24.305597 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:32:24.305597 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:32:24.305597 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:32:24.305597 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:32:24.305597 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:32:24.305597 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:32:24.305597 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:32:24.305597 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:32:24.305597 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:32:24.305597 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:32:24.305597 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:32:24.305597 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:32:24.306937 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:32:24.306937 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:32:24.306937 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:32:24.306937 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:32:24.306937 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:32:24.306937 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:32:24.306937 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:32:24.306937 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:32:24.306937 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:32:24.306937 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:32:24.306937 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:32:24.306937 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:32:24.306937 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:32:24.306937 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:32:24.306937 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:32:24.306937 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:32:24.306937 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:32:24.308382 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:32:24.308382 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:32:24.308382 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:32:24.308382 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 612/1203] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:32:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:32:24.313135 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79909df5-20f0-41de-b217-937ba2219cf3 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:24.415408 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79909df5-20f0-41de-b217-937ba2219cf3 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:24.445487 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:24.451291 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1938a1a-f2b0-4738-9e91-30d816d2dd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:24.451947 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1938a1a-f2b0-4738-9e91-30d816d2dd38 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:24.452230 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1938a1a-f2b0-4738-9e91-30d816d2dd38 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:24.452504 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1938a1a-f2b0-4738-9e91-30d816d2dd38 None admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:24.453175 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1938a1a-f2b0-4738-9e91-30d816d2dd38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:32:24.453480 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1938a1a-f2b0-4738-9e91-30d816d2dd38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:24.454920 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1938a1a-f2b0-4738-9e91-30d816d2dd38 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:24.466206 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 592/1204] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:32:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:32:24.472121 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a73cb9-39df-43f5-9f3b-d212b93e279a None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:24.527534 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:24.577422 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a73cb9-39df-43f5-9f3b-d212b93e279a None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:24.619024 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a678ea-cc4c-43b5-93d2-c93b59b2c558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:24.619627 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a678ea-cc4c-43b5-93d2-c93b59b2c558 None admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:24.619906 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a678ea-cc4c-43b5-93d2-c93b59b2c558 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:24.620176 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a678ea-cc4c-43b5-93d2-c93b59b2c558 None admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:24.620846 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a678ea-cc4c-43b5-93d2-c93b59b2c558 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:24.622266 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a678ea-cc4c-43b5-93d2-c93b59b2c558 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:24.726513 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 613/1205] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Aug 29 22:32:24 2022] POST /identity/v3/users => generated 251 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:32:26.708163 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6c660-d5a6-4e8d-9302-d9288dcdb1cd None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:26.709599 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6c660-d5a6-4e8d-9302-d9288dcdb1cd None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:26.709599 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6c660-d5a6-4e8d-9302-d9288dcdb1cd None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:26.710727 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 593/1206] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:32:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:32:26.716094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d8a40-84fe-412c-a2fa-7b6d00d98cbf None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:26.716534 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d8a40-84fe-412c-a2fa-7b6d00d98cbf None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:26.716637 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d8a40-84fe-412c-a2fa-7b6d00d98cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:26.735320 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-868d8a40-84fe-412c-a2fa-7b6d00d98cbf None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:32:26.736515 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-868d8a40-84fe-412c-a2fa-7b6d00d98cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:32:26.804668 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 614/1207] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:32:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:32:27.079670 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6c660-d5a6-4e8d-9302-d9288dcdb1cd None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:27.184377 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6c660-d5a6-4e8d-9302-d9288dcdb1cd None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:27.217978 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7c56bc-8eda-4f3f-8c4c-ca60ea568a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:27.218585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c56bc-8eda-4f3f-8c4c-ca60ea568a39 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:27.218859 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c56bc-8eda-4f3f-8c4c-ca60ea568a39 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:27.219125 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c56bc-8eda-4f3f-8c4c-ca60ea568a39 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:27.229669 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7c56bc-8eda-4f3f-8c4c-ca60ea568a39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:32:27.234523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-bb7c56bc-8eda-4f3f-8c4c-ca60ea568a39 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:32:27.234523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:32:27.234523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:32:27.234523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:32:27.234523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:32:27.234523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:32:27.234523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:32:27.234523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:32:27.234523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:32:27.234523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:32:27.234523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:32:27.234523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:32:27.234523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:32:27.234523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:32:27.234523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:32:27.234523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:32:27.235692 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:32:27.235692 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:32:27.235692 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:32:27.235692 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:32:27.235692 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:32:27.235692 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:32:27.235692 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:32:27.235692 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:32:27.235692 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:32:27.235692 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:32:27.235692 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:32:27.235692 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:32:27.235692 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:32:27.235692 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:32:27.235692 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:32:27.235692 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:32:27.236752 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:32:27.236752 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:32:27.236752 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:32:27.236752 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:32:27.236752 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:32:27.236752 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:32:27.236752 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:32:27.236752 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:32:27.236752 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:32:27.236752 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:32:27.236752 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:32:27.236752 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:32:27.236752 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 594/1208] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Aug 29 22:32:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:32:27.242405 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868d8a40-84fe-412c-a2fa-7b6d00d98cbf None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:27.286762 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:27.346377 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868d8a40-84fe-412c-a2fa-7b6d00d98cbf None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:27.380323 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3c875e-9f61-4af1-9ec5-224c428d2832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:27.380950 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c875e-9f61-4af1-9ec5-224c428d2832 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:27.381233 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c875e-9f61-4af1-9ec5-224c428d2832 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:27.381507 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c875e-9f61-4af1-9ec5-224c428d2832 None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:27.382160 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3c875e-9f61-4af1-9ec5-224c428d2832 None admin] RBAC: Adding query filter params (name=service) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:32:27.382462 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3c875e-9f61-4af1-9ec5-224c428d2832 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:27.383816 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3c875e-9f61-4af1-9ec5-224c428d2832 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:27.392317 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 615/1209] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Aug 29 22:32:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:32:27.398231 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7c56bc-8eda-4f3f-8c4c-ca60ea568a39 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:27.449030 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:27.500151 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7c56bc-8eda-4f3f-8c4c-ca60ea568a39 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:27.532931 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d457314-18ab-4040-ada1-821c654e2e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:27.533565 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d457314-18ab-4040-ada1-821c654e2e01 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:27.533853 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d457314-18ab-4040-ada1-821c654e2e01 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:27.534128 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d457314-18ab-4040-ada1-821c654e2e01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:27.546121 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d457314-18ab-4040-ada1-821c654e2e01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:27.547564 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d457314-18ab-4040-ada1-821c654e2e01 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:27.563721 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-1d457314-18ab-4040-ada1-821c654e2e01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:32:27.563721 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:32:27.563721 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:32:27.563721 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:32:27.563721 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:32:27.563721 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:32:27.563721 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:32:27.563721 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:32:27.563721 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:32:27.563721 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:32:27.563721 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:32:27.563721 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:32:27.563721 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:32:27.563721 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:32:27.563721 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:32:27.563721 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:32:27.564816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:32:27.564816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:32:27.564816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:32:27.564816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:32:27.564816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:32:27.564816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:32:27.564816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:32:27.564816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:32:27.564816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:32:27.564816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:32:27.564816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:32:27.564816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:32:27.564816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:32:27.564816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:32:27.564816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:32:27.564816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:32:27.564816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:32:27.566237 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:32:27.566237 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:32:27.566237 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:32:27.566237 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 595/1210] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:32:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:32:27.570875 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3c875e-9f61-4af1-9ec5-224c428d2832 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:27.675302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:27.679424 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3c875e-9f61-4af1-9ec5-224c428d2832 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:27.713100 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0b5390-2a84-4253-af02-884d5168c2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:27.713697 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b5390-2a84-4253-af02-884d5168c2ab None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:27.713979 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b5390-2a84-4253-af02-884d5168c2ab None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:27.714254 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b5390-2a84-4253-af02-884d5168c2ab None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:27.714887 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0b5390-2a84-4253-af02-884d5168c2ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:32:27.715190 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0b5390-2a84-4253-af02-884d5168c2ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:27.716558 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0b5390-2a84-4253-af02-884d5168c2ab None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:27.729286 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 616/1211] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:32:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:32:27.735107 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d457314-18ab-4040-ada1-821c654e2e01 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:27.837038 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d457314-18ab-4040-ada1-821c654e2e01 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:27.838602 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:27.872776 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a2422-9128-438e-b748-cb46b935bcd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:27.873535 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a2422-9128-438e-b748-cb46b935bcd4 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:27.874016 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a2422-9128-438e-b748-cb46b935bcd4 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:27.874443 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a2422-9128-438e-b748-cb46b935bcd4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:27.882478 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a2422-9128-438e-b748-cb46b935bcd4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:27.883966 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a2422-9128-438e-b748-cb46b935bcd4 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:27.895797 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-e86a2422-9128-438e-b748-cb46b935bcd4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 22:32:27.895797 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:32:27.895797 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:32:27.895797 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:32:27.895797 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:32:27.895797 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:32:27.895797 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:32:27.895797 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:32:27.895797 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:32:27.895797 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:32:27.895797 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:32:27.895797 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:32:27.895797 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:32:27.895797 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:32:27.895797 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:32:27.895797 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:32:27.896914 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:32:27.896914 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:32:27.896914 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:32:27.896914 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:32:27.896914 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:32:27.896914 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:32:27.896914 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:32:27.896914 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:32:27.896914 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:32:27.896914 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:32:27.896914 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:32:27.896914 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:32:27.896914 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:32:27.896914 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:32:27.896914 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:32:27.896914 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:32:27.896914 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:32:27.896914 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:32:27.898028 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:32:27.898028 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:32:27.898028 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:32:27.898028 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:32:27.898028 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:32:27.898028 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:32:27.898028 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:32:27.898028 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:32:27.898028 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 22:32:27.898028 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:32:27.898028 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 596/1212] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:32:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:32:27.902950 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0b5390-2a84-4253-af02-884d5168c2ab None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:28.005490 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0b5390-2a84-4253-af02-884d5168c2ab None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:28.041767 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b45628-ea8e-4b9d-8218-169b5433e2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:28.042365 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b45628-ea8e-4b9d-8218-169b5433e2f3 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:28.042642 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b45628-ea8e-4b9d-8218-169b5433e2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:28.042916 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b45628-ea8e-4b9d-8218-169b5433e2f3 None admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:28.043638 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b45628-ea8e-4b9d-8218-169b5433e2f3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:32:28.043940 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b45628-ea8e-4b9d-8218-169b5433e2f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:28.045448 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b45628-ea8e-4b9d-8218-169b5433e2f3 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:28.062453 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 617/1213] 10.222.0.52 () {56 vars in 1143 bytes} [Mon Aug 29 22:32:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:32:28.069555 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a2422-9128-438e-b748-cb46b935bcd4 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:28.171938 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a2422-9128-438e-b748-cb46b935bcd4 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:28.206317 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42fa1f9-df6d-4207-9864-8e5e74f94a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:28.206912 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42fa1f9-df6d-4207-9864-8e5e74f94a0c None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:28.207185 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42fa1f9-df6d-4207-9864-8e5e74f94a0c None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:28.208298 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42fa1f9-df6d-4207-9864-8e5e74f94a0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:28.237476 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:28.274431 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42fa1f9-df6d-4207-9864-8e5e74f94a0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:28.275794 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42fa1f9-df6d-4207-9864-8e5e74f94a0c None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:28.375441 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 597/1214] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:32:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 22:32:28.381586 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b45628-ea8e-4b9d-8218-169b5433e2f3 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:28.401128 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:28.482125 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b45628-ea8e-4b9d-8218-169b5433e2f3 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:28.514804 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fabd35-db71-4fa6-ad63-a3c1c95a0fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:28.515423 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fabd35-db71-4fa6-ad63-a3c1c95a0fb1 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:28.515704 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fabd35-db71-4fa6-ad63-a3c1c95a0fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:28.515976 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fabd35-db71-4fa6-ad63-a3c1c95a0fb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:28.527658 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fabd35-db71-4fa6-ad63-a3c1c95a0fb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:28.529078 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fabd35-db71-4fa6-ad63-a3c1c95a0fb1 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:28.545329 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-61fabd35-db71-4fa6-ad63-a3c1c95a0fb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:32:28.545329 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:32:28.545329 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:32:28.545329 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:32:28.545329 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:32:28.545329 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:32:28.545329 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:32:28.545329 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:32:28.545329 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:32:28.545329 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:32:28.545329 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:32:28.545329 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:32:28.545329 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:32:28.545329 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:32:28.545329 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:32:28.545329 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:32:28.552354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:32:28.552354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:32:28.552354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:32:28.552354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:32:28.552354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:32:28.552354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:32:28.552354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:32:28.552354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:32:28.552354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:32:28.552354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:32:28.552354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:32:28.552354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:32:28.552354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:32:28.552354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:32:28.552354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:32:28.552354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:32:28.552354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:32:28.553651 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:32:28.553651 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:32:28.553651 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:32:28.553651 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 618/1215] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:32:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:32:28.553922 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42fa1f9-df6d-4207-9864-8e5e74f94a0c None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:28.613724 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:28.651936 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42fa1f9-df6d-4207-9864-8e5e74f94a0c None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:28.687178 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d4b82d-f962-46fe-9d57-9f5ac6d33e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:28.687782 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4b82d-f962-46fe-9d57-9f5ac6d33e7d None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:28.688062 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4b82d-f962-46fe-9d57-9f5ac6d33e7d None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:28.688332 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4b82d-f962-46fe-9d57-9f5ac6d33e7d None admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:28.688995 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d4b82d-f962-46fe-9d57-9f5ac6d33e7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:32:28.689296 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d4b82d-f962-46fe-9d57-9f5ac6d33e7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:28.690755 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d4b82d-f962-46fe-9d57-9f5ac6d33e7d None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:28.702096 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 598/1216] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:32:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:32:28.708600 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fabd35-db71-4fa6-ad63-a3c1c95a0fb1 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:28.778947 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:28.810916 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fabd35-db71-4fa6-ad63-a3c1c95a0fb1 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:28.845986 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af5e68-9dcf-42dd-ad47-879e34bf35f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:28.846610 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af5e68-9dcf-42dd-ad47-879e34bf35f0 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:28.846892 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af5e68-9dcf-42dd-ad47-879e34bf35f0 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:28.847165 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af5e68-9dcf-42dd-ad47-879e34bf35f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:28.853778 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15af5e68-9dcf-42dd-ad47-879e34bf35f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:28.855320 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15af5e68-9dcf-42dd-ad47-879e34bf35f0 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:28.866837 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-15af5e68-9dcf-42dd-ad47-879e34bf35f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:32:28.866837 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:32:28.866837 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:32:28.866837 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:32:28.866837 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:32:28.866837 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:32:28.866837 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:32:28.866837 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:32:28.866837 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:32:28.866837 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:32:28.866837 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:32:28.866837 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:32:28.866837 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:32:28.866837 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:32:28.866837 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:32:28.867929 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:32:28.867929 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:32:28.867929 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:32:28.867929 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:32:28.867929 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:32:28.867929 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:32:28.867929 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:32:28.867929 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:32:28.867929 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:32:28.867929 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:32:28.867929 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:32:28.867929 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:32:28.867929 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:32:28.867929 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:32:28.867929 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:32:28.867929 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:32:28.867929 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:32:28.868916 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:32:28.868916 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:32:28.868916 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:32:28.868916 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:32:28.868916 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:32:28.868916 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:32:28.868916 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:32:28.868916 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:32:28.868916 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 619/1217] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:32:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:32:28.873949 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d4b82d-f962-46fe-9d57-9f5ac6d33e7d None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:28.984737 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d4b82d-f962-46fe-9d57-9f5ac6d33e7d None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:29.008813 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:29.020533 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e0049-4c83-446f-a37c-6d498eb7586a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:29.021194 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e0049-4c83-446f-a37c-6d498eb7586a None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:29.021469 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e0049-4c83-446f-a37c-6d498eb7586a None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:29.021782 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e0049-4c83-446f-a37c-6d498eb7586a None admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:29.022598 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0e0049-4c83-446f-a37c-6d498eb7586a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:32:29.022924 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0e0049-4c83-446f-a37c-6d498eb7586a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:29.024387 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0e0049-4c83-446f-a37c-6d498eb7586a None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:29.038922 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 599/1218] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Aug 29 22:32:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:32:29.045146 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af5e68-9dcf-42dd-ad47-879e34bf35f0 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:29.146615 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af5e68-9dcf-42dd-ad47-879e34bf35f0 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:29.174710 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:29.180794 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7a1e0d-6642-421e-a549-91d95271d7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:29.181391 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a1e0d-6642-421e-a549-91d95271d7bd None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:29.181670 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a1e0d-6642-421e-a549-91d95271d7bd None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:29.181993 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a1e0d-6642-421e-a549-91d95271d7bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:29.182709 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7a1e0d-6642-421e-a549-91d95271d7bd None admin] RBAC: Adding query filter params (role.id=db83c2f8e21945c3ab9642c620d29a23, scope.project.id=ee84c3340f2641ad963655a7a212ef96, user.id=d8fd0f6e5b144d2ca78b8d96722502a7) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:32:29.183017 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7a1e0d-6642-421e-a549-91d95271d7bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:29.184439 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7a1e0d-6642-421e-a549-91d95271d7bd None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:29.190374 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 620/1219] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:32:29 2022] GET /identity/v3/role_assignments?user.id=d8fd0f6e5b144d2ca78b8d96722502a7&scope.project.id=ee84c3340f2641ad963655a7a212ef96&role.id=db83c2f8e21945c3ab9642c620d29a23 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:32:31.098381 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912f4fb7-54cd-4f86-ac6e-7bdff8eda83a None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:31.098915 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912f4fb7-54cd-4f86-ac6e-7bdff8eda83a None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:31.099026 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912f4fb7-54cd-4f86-ac6e-7bdff8eda83a None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:31.100311 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 600/1220] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:32:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:32:31.105731 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900a050-221b-4da1-b7c6-cfb9bda148d8 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:31.106161 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900a050-221b-4da1-b7c6-cfb9bda148d8 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:31.106293 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900a050-221b-4da1-b7c6-cfb9bda148d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:31.124191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-e900a050-221b-4da1-b7c6-cfb9bda148d8 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:32:31.125425 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-e900a050-221b-4da1-b7c6-cfb9bda148d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:32:31.193054 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 621/1221] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:32:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:32:31.468380 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912f4fb7-54cd-4f86-ac6e-7bdff8eda83a None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:31.579744 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912f4fb7-54cd-4f86-ac6e-7bdff8eda83a None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:31.615067 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0ec445-acb0-43b7-a23f-8eb3ac7742f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:31.615753 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ec445-acb0-43b7-a23f-8eb3ac7742f0 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:31.616027 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ec445-acb0-43b7-a23f-8eb3ac7742f0 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:31.616299 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ec445-acb0-43b7-a23f-8eb3ac7742f0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:31.626464 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0ec445-acb0-43b7-a23f-8eb3ac7742f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:32:31.635097 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-3f0ec445-acb0-43b7-a23f-8eb3ac7742f0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:32:31.635097 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:32:31.635097 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:32:31.635097 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:32:31.635097 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:32:31.635097 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:32:31.635097 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:32:31.635097 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:32:31.635097 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:32:31.635097 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:32:31.635097 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:32:31.635097 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:32:31.635097 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:32:31.635097 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:32:31.635097 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:32:31.635097 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:32:31.636303 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:32:31.636303 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:32:31.636303 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:32:31.636303 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:32:31.636303 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:32:31.636303 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:32:31.636303 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:32:31.636303 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:32:31.636303 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:32:31.636303 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:32:31.636303 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:32:31.636303 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:32:31.636303 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:32:31.636303 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:32:31.636303 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:32:31.636303 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:32:31.637309 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:32:31.637309 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:32:31.637309 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:32:31.637309 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:32:31.637309 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:32:31.637309 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:32:31.637309 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:32:31.637309 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:32:31.637309 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:32:31.637309 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:32:31.637309 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:32:31.637309 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:32:31.637309 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 601/1222] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Aug 29 22:32:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:32:31.639316 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900a050-221b-4da1-b7c6-cfb9bda148d8 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:31.739504 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900a050-221b-4da1-b7c6-cfb9bda148d8 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:31.763645 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:31.774880 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29666187-d180-4243-a3de-9d9ec25d6c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:31.775488 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29666187-d180-4243-a3de-9d9ec25d6c7d None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:31.775768 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29666187-d180-4243-a3de-9d9ec25d6c7d None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:31.776050 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29666187-d180-4243-a3de-9d9ec25d6c7d None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:31.776711 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29666187-d180-4243-a3de-9d9ec25d6c7d None admin] RBAC: Adding query filter params (name=service) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:32:31.777035 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29666187-d180-4243-a3de-9d9ec25d6c7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:31.778402 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29666187-d180-4243-a3de-9d9ec25d6c7d None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:31.787148 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 622/1223] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Aug 29 22:32:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:32:31.793376 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0ec445-acb0-43b7-a23f-8eb3ac7742f0 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:31.844770 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:31.904726 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0ec445-acb0-43b7-a23f-8eb3ac7742f0 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:31.939738 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbd959c-d9bf-4e55-8b8f-cfb389d00e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:31.940370 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd959c-d9bf-4e55-8b8f-cfb389d00e4d None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:31.940652 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd959c-d9bf-4e55-8b8f-cfb389d00e4d None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:31.946610 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd959c-d9bf-4e55-8b8f-cfb389d00e4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:31.951123 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbd959c-d9bf-4e55-8b8f-cfb389d00e4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:31.952584 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbd959c-d9bf-4e55-8b8f-cfb389d00e4d None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:31.967334 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-4fbd959c-d9bf-4e55-8b8f-cfb389d00e4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:32:31.967334 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:32:31.967334 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:32:31.967334 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:32:31.967334 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:32:31.967334 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:32:31.967334 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:32:31.967334 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:32:31.967334 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:32:31.967334 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:32:31.967334 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:32:31.967334 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:32:31.967334 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:32:31.967334 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:32:31.967334 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:32:31.967334 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:32:31.968448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:32:31.968448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:32:31.968448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:32:31.968448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:32:31.968448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:32:31.968448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:32:31.968448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:32:31.968448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:32:31.968448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:32:31.968448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:32:31.968448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:32:31.968448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:32:31.968448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:32:31.968448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:32:31.968448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:32:31.968448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:32:31.968448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:32:31.969757 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:32:31.969757 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:32:31.969757 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:32:31.969757 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 602/1224] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:32:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:32:31.974991 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29666187-d180-4243-a3de-9d9ec25d6c7d None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:32.078648 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29666187-d180-4243-a3de-9d9ec25d6c7d None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:32.112045 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd5f48b-cb12-4d8d-847a-ec1e0d007976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:32.112684 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5f48b-cb12-4d8d-847a-ec1e0d007976 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:32.112930 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5f48b-cb12-4d8d-847a-ec1e0d007976 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:32.113249 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5f48b-cb12-4d8d-847a-ec1e0d007976 None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:32.114613 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd5f48b-cb12-4d8d-847a-ec1e0d007976 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:32:32.114943 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd5f48b-cb12-4d8d-847a-ec1e0d007976 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:32.116289 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd5f48b-cb12-4d8d-847a-ec1e0d007976 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:32.128058 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 623/1225] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:32:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:32:32.133905 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbd959c-d9bf-4e55-8b8f-cfb389d00e4d None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:32.234402 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:32.241688 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbd959c-d9bf-4e55-8b8f-cfb389d00e4d None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:32.276281 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ccf543-05ab-4cff-8ec2-262e11405a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:32.276936 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ccf543-05ab-4cff-8ec2-262e11405a2a None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:32.277216 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ccf543-05ab-4cff-8ec2-262e11405a2a None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:32.277491 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ccf543-05ab-4cff-8ec2-262e11405a2a None admin] PATH_INFO: `/v3/users/placement` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:32.284956 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ccf543-05ab-4cff-8ec2-262e11405a2a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:32.287745 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ccf543-05ab-4cff-8ec2-262e11405a2a None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:32.299304 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-e3ccf543-05ab-4cff-8ec2-262e11405a2a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 22:32:32.299304 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:32:32.299304 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:32:32.299304 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:32:32.299304 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:32:32.299304 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:32:32.299304 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:32:32.299304 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:32:32.299304 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:32:32.299304 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:32:32.299304 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:32:32.299304 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:32:32.299304 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:32:32.299304 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:32:32.299304 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:32:32.299304 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:32:32.300504 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:32:32.300504 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:32:32.300504 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:32:32.300504 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:32:32.300504 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:32:32.300504 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:32:32.300504 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:32:32.300504 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:32:32.300504 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:32:32.300504 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:32:32.300504 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:32:32.300504 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:32:32.300504 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:32:32.300504 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:32:32.300504 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:32:32.300504 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:32:32.300504 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:32:32.300504 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:32:32.301633 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:32:32.301633 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:32:32.301633 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:32:32.301633 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:32:32.301633 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:32:32.301633 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:32:32.301633 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:32:32.301633 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:32:32.301633 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 22:32:32.301633 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:32:32.301633 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 603/1226] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:32:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:32:32.306668 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd5f48b-cb12-4d8d-847a-ec1e0d007976 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:32.354558 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:32.408032 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd5f48b-cb12-4d8d-847a-ec1e0d007976 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:32.444488 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5874c4bf-aa63-4db0-aec1-49edc0fe6704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:32.445415 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5874c4bf-aa63-4db0-aec1-49edc0fe6704 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:32.445692 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5874c4bf-aa63-4db0-aec1-49edc0fe6704 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:32.445971 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5874c4bf-aa63-4db0-aec1-49edc0fe6704 None admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:32.446696 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5874c4bf-aa63-4db0-aec1-49edc0fe6704 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:32:32.447001 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5874c4bf-aa63-4db0-aec1-49edc0fe6704 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:32.448425 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5874c4bf-aa63-4db0-aec1-49edc0fe6704 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:32.466483 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 624/1227] 10.222.0.52 () {56 vars in 1143 bytes} [Mon Aug 29 22:32:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:32:32.472267 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ccf543-05ab-4cff-8ec2-262e11405a2a None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:32.579031 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ccf543-05ab-4cff-8ec2-262e11405a2a None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:32.613275 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1220832b-5fe6-4b36-b2dc-3bdaa65d61b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:32.613975 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1220832b-5fe6-4b36-b2dc-3bdaa65d61b6 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:32.614307 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1220832b-5fe6-4b36-b2dc-3bdaa65d61b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:32.614587 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1220832b-5fe6-4b36-b2dc-3bdaa65d61b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:32.625746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1220832b-5fe6-4b36-b2dc-3bdaa65d61b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:32.627191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1220832b-5fe6-4b36-b2dc-3bdaa65d61b6 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:32.642128 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-1220832b-5fe6-4b36-b2dc-3bdaa65d61b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:32:32.642128 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:32:32.642128 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:32:32.642128 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:32:32.642128 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:32:32.642128 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:32:32.642128 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:32:32.642128 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:32:32.642128 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:32:32.642128 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:32:32.642128 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:32:32.642128 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:32:32.642128 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:32:32.642128 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:32:32.642128 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:32:32.642128 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:32:32.643253 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:32:32.643253 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:32:32.643253 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:32:32.643253 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:32:32.643253 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:32:32.643253 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:32:32.643253 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:32:32.643253 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:32:32.643253 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:32:32.643253 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:32:32.643253 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:32:32.643253 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:32:32.643253 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:32:32.643253 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:32:32.643253 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:32:32.643253 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:32:32.643253 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:32:32.644156 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:32:32.644156 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:32:32.644156 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:32:32.644156 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 604/1228] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:32:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:32:32.650082 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5874c4bf-aa63-4db0-aec1-49edc0fe6704 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:32.751343 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5874c4bf-aa63-4db0-aec1-49edc0fe6704 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:32.753646 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:32.786888 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b4a00-953e-4a7b-ac0a-24cd94c122c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:32.787486 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b4a00-953e-4a7b-ac0a-24cd94c122c7 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:32.787762 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b4a00-953e-4a7b-ac0a-24cd94c122c7 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:32.788032 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b4a00-953e-4a7b-ac0a-24cd94c122c7 None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:32.788661 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b4a00-953e-4a7b-ac0a-24cd94c122c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:32:32.788990 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b4a00-953e-4a7b-ac0a-24cd94c122c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:32.790336 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b4a00-953e-4a7b-ac0a-24cd94c122c7 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:32.801654 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 625/1229] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:32:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:32:32.807915 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1220832b-5fe6-4b36-b2dc-3bdaa65d61b6 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:32.818347 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:32.908975 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1220832b-5fe6-4b36-b2dc-3bdaa65d61b6 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:32.943630 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9aa00e-0732-46fe-affc-75ed0182124c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:32.944257 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9aa00e-0732-46fe-affc-75ed0182124c None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:32.944536 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9aa00e-0732-46fe-affc-75ed0182124c None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:32.944832 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9aa00e-0732-46fe-affc-75ed0182124c None admin] PATH_INFO: `/v3/projects/service` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:32.951591 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9aa00e-0732-46fe-affc-75ed0182124c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:32.953185 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9aa00e-0732-46fe-affc-75ed0182124c None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:32.965405 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-fc9aa00e-0732-46fe-affc-75ed0182124c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:32:32.965405 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:32:32.965405 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:32:32.965405 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:32:32.965405 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:32:32.965405 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:32:32.965405 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:32:32.965405 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:32:32.965405 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:32:32.965405 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:32:32.965405 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:32:32.965405 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:32:32.965405 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:32:32.965405 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:32:32.965405 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:32:32.966467 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:32:32.966467 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:32:32.966467 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:32:32.966467 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:32:32.966467 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:32:32.966467 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:32:32.966467 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:32:32.966467 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:32:32.966467 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:32:32.966467 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:32:32.966467 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:32:32.966467 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:32:32.966467 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:32:32.966467 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:32:32.966467 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:32:32.966467 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:32:32.966467 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:32:32.967388 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:32:32.967388 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:32:32.967388 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:32:32.967388 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:32:32.967388 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:32:32.967388 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:32:32.967388 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:32:32.967388 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:32:32.967388 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 605/1230] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:32:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:32:32.972639 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b4a00-953e-4a7b-ac0a-24cd94c122c7 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:33.076196 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b4a00-953e-4a7b-ac0a-24cd94c122c7 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:33.110162 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4f7b6-fbd3-4063-8813-c9ac956d8b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:33.110768 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4f7b6-fbd3-4063-8813-c9ac956d8b43 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:33.111089 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4f7b6-fbd3-4063-8813-c9ac956d8b43 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:33.111368 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4f7b6-fbd3-4063-8813-c9ac956d8b43 None admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:33.112062 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf4f7b6-fbd3-4063-8813-c9ac956d8b43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:32:33.112362 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf4f7b6-fbd3-4063-8813-c9ac956d8b43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:33.114417 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf4f7b6-fbd3-4063-8813-c9ac956d8b43 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:33.127660 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 626/1231] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Aug 29 22:32:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:32:33.134576 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9aa00e-0732-46fe-affc-75ed0182124c None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:33.201859 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:33.236066 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9aa00e-0732-46fe-affc-75ed0182124c None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:33.271466 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e92f8-4700-48e9-9044-ece760793433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:33.272115 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e92f8-4700-48e9-9044-ece760793433 None admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:33.272426 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e92f8-4700-48e9-9044-ece760793433 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:33.272702 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e92f8-4700-48e9-9044-ece760793433 None admin] PATH_INFO: `/v3/projects/ee84c3340f2641ad963655a7a212ef96/users/d8fd0f6e5b144d2ca78b8d96722502a7/roles/db83c2f8e21945c3ab9642c620d29a23` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:33.288214 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97e92f8-4700-48e9-9044-ece760793433 None admin] RBAC: Authorizing `identity:create_grant(project_id=ee84c3340f2641ad963655a7a212ef96, user_id=d8fd0f6e5b144d2ca78b8d96722502a7, role_id=db83c2f8e21945c3ab9642c620d29a23)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:33.289737 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97e92f8-4700-48e9-9044-ece760793433 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:33.446836 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 606/1232] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Aug 29 22:32:33 2022] PUT /identity/v3/projects/ee84c3340f2641ad963655a7a212ef96/users/d8fd0f6e5b144d2ca78b8d96722502a7/roles/db83c2f8e21945c3ab9642c620d29a23 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:32:35.408356 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08271eaf-eae5-4616-bb3d-c67b1853bc2f None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:35.409002 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08271eaf-eae5-4616-bb3d-c67b1853bc2f None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:35.409107 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08271eaf-eae5-4616-bb3d-c67b1853bc2f None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:35.410535 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 627/1233] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:32:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:32:35.415917 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7287dec1-1c48-4010-bf31-1d393d32836b None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:35.416346 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7287dec1-1c48-4010-bf31-1d393d32836b None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:35.416450 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7287dec1-1c48-4010-bf31-1d393d32836b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:35.435130 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-7287dec1-1c48-4010-bf31-1d393d32836b None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:32:35.436303 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-7287dec1-1c48-4010-bf31-1d393d32836b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:32:35.510714 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 607/1234] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:32:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:32:35.787989 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08271eaf-eae5-4616-bb3d-c67b1853bc2f None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:35.815094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:35.890806 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08271eaf-eae5-4616-bb3d-c67b1853bc2f None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:35.925089 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef08756-d3e8-4eea-a232-1f909c05efc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:35.925712 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef08756-d3e8-4eea-a232-1f909c05efc0 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:35.926130 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef08756-d3e8-4eea-a232-1f909c05efc0 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:35.926411 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef08756-d3e8-4eea-a232-1f909c05efc0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:35.938012 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef08756-d3e8-4eea-a232-1f909c05efc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:32:35.943001 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-2ef08756-d3e8-4eea-a232-1f909c05efc0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:32:35.943001 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:32:35.943001 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:32:35.943001 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:32:35.943001 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:32:35.943001 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:32:35.943001 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:32:35.943001 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:32:35.943001 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:32:35.943001 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:32:35.943001 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:32:35.943001 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:32:35.943001 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:32:35.943001 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:32:35.943001 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:32:35.943001 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:32:35.944134 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:32:35.944134 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:32:35.944134 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:32:35.944134 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:32:35.944134 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:32:35.944134 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:32:35.944134 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:32:35.944134 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:32:35.944134 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:32:35.944134 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:32:35.944134 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:32:35.944134 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:32:35.944134 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:32:35.944134 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:32:35.944134 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:32:35.944134 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:32:35.945484 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:32:35.945484 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:32:35.945484 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:32:35.945484 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:32:35.945484 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:32:35.945484 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:32:35.945484 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:32:35.945484 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:32:35.945484 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:32:35.945484 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:32:35.945484 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:32:35.945484 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:32:35.945484 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 628/1235] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Aug 29 22:32:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:32:35.950343 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7287dec1-1c48-4010-bf31-1d393d32836b None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:35.997228 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:36.053979 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7287dec1-1c48-4010-bf31-1d393d32836b None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:36.092919 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51a56d2-b29f-438d-8a7b-71efb2c805b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:36.093548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a56d2-b29f-438d-8a7b-71efb2c805b4 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:36.093811 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a56d2-b29f-438d-8a7b-71efb2c805b4 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:36.094087 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a56d2-b29f-438d-8a7b-71efb2c805b4 None admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:36.094785 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51a56d2-b29f-438d-8a7b-71efb2c805b4 None admin] RBAC: Adding query filter params (name=service) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:32:36.095057 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51a56d2-b29f-438d-8a7b-71efb2c805b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:36.096435 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51a56d2-b29f-438d-8a7b-71efb2c805b4 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:36.105633 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 608/1236] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Aug 29 22:32:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:32:36.111788 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef08756-d3e8-4eea-a232-1f909c05efc0 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:36.199430 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:36.215259 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef08756-d3e8-4eea-a232-1f909c05efc0 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:36.249496 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d491112-240e-4ce6-a572-3e8d54f7eceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:36.250341 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d491112-240e-4ce6-a572-3e8d54f7eceb None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:36.250758 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d491112-240e-4ce6-a572-3e8d54f7eceb None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:36.251074 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d491112-240e-4ce6-a572-3e8d54f7eceb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:36.262046 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d491112-240e-4ce6-a572-3e8d54f7eceb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:36.263556 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d491112-240e-4ce6-a572-3e8d54f7eceb None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:36.279345 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-0d491112-240e-4ce6-a572-3e8d54f7eceb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:32:36.279345 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:32:36.279345 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:32:36.279345 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:32:36.279345 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:32:36.279345 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:32:36.279345 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:32:36.279345 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:32:36.279345 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:32:36.279345 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:32:36.279345 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:32:36.279345 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:32:36.279345 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:32:36.279345 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:32:36.279345 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:32:36.279345 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:32:36.280517 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:32:36.280517 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:32:36.280517 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:32:36.280517 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:32:36.280517 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:32:36.280517 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:32:36.280517 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:32:36.280517 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:32:36.280517 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:32:36.280517 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:32:36.280517 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:32:36.280517 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:32:36.280517 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:32:36.280517 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:32:36.280517 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:32:36.280517 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:32:36.280517 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:32:36.281483 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:32:36.281483 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:32:36.281483 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:32:36.281483 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 629/1237] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:32:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:32:36.287732 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51a56d2-b29f-438d-8a7b-71efb2c805b4 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:36.389867 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51a56d2-b29f-438d-8a7b-71efb2c805b4 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:36.399587 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:36.424313 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979624d6-6c5c-4904-b325-4c98630e83b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:36.425000 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979624d6-6c5c-4904-b325-4c98630e83b8 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:36.425205 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979624d6-6c5c-4904-b325-4c98630e83b8 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:36.425476 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979624d6-6c5c-4904-b325-4c98630e83b8 None admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:36.426179 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979624d6-6c5c-4904-b325-4c98630e83b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:32:36.426482 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979624d6-6c5c-4904-b325-4c98630e83b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:36.428114 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979624d6-6c5c-4904-b325-4c98630e83b8 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:36.440051 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 609/1238] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:32:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:32:36.446773 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d491112-240e-4ce6-a572-3e8d54f7eceb None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:36.552092 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d491112-240e-4ce6-a572-3e8d54f7eceb None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:36.587479 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec05cf64-d911-49f1-b133-4280862cb2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:36.588092 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05cf64-d911-49f1-b133-4280862cb2fd None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:36.588376 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05cf64-d911-49f1-b133-4280862cb2fd None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:36.588641 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05cf64-d911-49f1-b133-4280862cb2fd None admin] PATH_INFO: `/v3/users/placement` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:36.591274 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:36.596154 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec05cf64-d911-49f1-b133-4280862cb2fd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:36.598000 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec05cf64-d911-49f1-b133-4280862cb2fd None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:36.608735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-ec05cf64-d911-49f1-b133-4280862cb2fd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 22:32:36.608735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:32:36.608735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:32:36.608735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:32:36.608735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:32:36.608735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:32:36.608735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:32:36.608735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:32:36.608735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:32:36.608735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:32:36.608735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:32:36.608735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:32:36.608735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:32:36.608735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:32:36.608735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:32:36.608735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:32:36.610062 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:32:36.610062 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:32:36.610062 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:32:36.610062 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:32:36.610062 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:32:36.610062 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:32:36.610062 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:32:36.610062 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:32:36.610062 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:32:36.610062 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:32:36.610062 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:32:36.610062 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:32:36.610062 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:32:36.610062 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:32:36.610062 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:32:36.610062 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:32:36.610062 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:32:36.610062 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:32:36.611453 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:32:36.611453 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:32:36.611453 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:32:36.611453 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:32:36.611453 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:32:36.611453 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:32:36.611453 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:32:36.611453 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:32:36.611453 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 22:32:36.611453 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:32:36.611453 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 630/1239] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:32:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:32:36.615830 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979624d6-6c5c-4904-b325-4c98630e83b8 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:36.719071 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979624d6-6c5c-4904-b325-4c98630e83b8 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:36.753970 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4239168f-9690-454c-9901-ea4763179a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:36.755294 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4239168f-9690-454c-9901-ea4763179a83 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:36.755618 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4239168f-9690-454c-9901-ea4763179a83 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:36.755936 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4239168f-9690-454c-9901-ea4763179a83 None admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:36.756749 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4239168f-9690-454c-9901-ea4763179a83 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:32:36.757116 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4239168f-9690-454c-9901-ea4763179a83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:36.758634 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4239168f-9690-454c-9901-ea4763179a83 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:36.775964 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 610/1240] 10.222.0.52 () {56 vars in 1143 bytes} [Mon Aug 29 22:32:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:32:36.781578 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec05cf64-d911-49f1-b133-4280862cb2fd None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:36.882452 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec05cf64-d911-49f1-b133-4280862cb2fd None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:36.916082 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81186bb-a638-4f5b-934c-f06a4a7bf2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:36.916697 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81186bb-a638-4f5b-934c-f06a4a7bf2d2 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:36.917007 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81186bb-a638-4f5b-934c-f06a4a7bf2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:36.917279 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81186bb-a638-4f5b-934c-f06a4a7bf2d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:36.980444 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:36.987063 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81186bb-a638-4f5b-934c-f06a4a7bf2d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:36.990063 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81186bb-a638-4f5b-934c-f06a4a7bf2d2 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:37.092648 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 631/1241] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:32:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 22:32:37.099373 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4239168f-9690-454c-9901-ea4763179a83 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:37.128349 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:37.200851 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4239168f-9690-454c-9901-ea4763179a83 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:37.234702 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2d6a22-5589-4100-b84c-9d61d703cd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:37.235255 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d6a22-5589-4100-b84c-9d61d703cd9b None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:37.235534 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d6a22-5589-4100-b84c-9d61d703cd9b None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:37.235809 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d6a22-5589-4100-b84c-9d61d703cd9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:37.247693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2d6a22-5589-4100-b84c-9d61d703cd9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:37.249216 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2d6a22-5589-4100-b84c-9d61d703cd9b None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:37.264082 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-3a2d6a22-5589-4100-b84c-9d61d703cd9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:32:37.264082 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:32:37.264082 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:32:37.264082 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:32:37.264082 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:32:37.264082 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:32:37.264082 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:32:37.264082 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:32:37.264082 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:32:37.264082 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:32:37.264082 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:32:37.264082 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:32:37.264082 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:32:37.264082 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:32:37.264082 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:32:37.264082 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:32:37.265267 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:32:37.265267 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:32:37.265267 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:32:37.265267 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:32:37.265267 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:32:37.265267 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:32:37.265267 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:32:37.265267 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:32:37.265267 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:32:37.265267 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:32:37.265267 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:32:37.265267 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:32:37.265267 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:32:37.265267 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:32:37.265267 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:32:37.265267 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:32:37.265267 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:32:37.266580 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:32:37.266580 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:32:37.266580 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:32:37.266580 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 611/1242] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:32:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:32:37.271063 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81186bb-a638-4f5b-934c-f06a4a7bf2d2 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:37.367375 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:37.373109 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81186bb-a638-4f5b-934c-f06a4a7bf2d2 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:37.407994 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b88d03-ae11-4b21-ae3c-5bb93ae62a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:37.416485 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b88d03-ae11-4b21-ae3c-5bb93ae62a7b None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:37.416485 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b88d03-ae11-4b21-ae3c-5bb93ae62a7b None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:37.416485 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b88d03-ae11-4b21-ae3c-5bb93ae62a7b None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:37.416485 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b88d03-ae11-4b21-ae3c-5bb93ae62a7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:32:37.416485 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b88d03-ae11-4b21-ae3c-5bb93ae62a7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:37.416485 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b88d03-ae11-4b21-ae3c-5bb93ae62a7b None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:37.424730 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 632/1243] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:32:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:32:37.431416 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2d6a22-5589-4100-b84c-9d61d703cd9b None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:37.511035 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:37.532989 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2d6a22-5589-4100-b84c-9d61d703cd9b None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:37.568689 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da25d0-46c2-4b21-948e-b0d5a5397110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:37.569346 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da25d0-46c2-4b21-948e-b0d5a5397110 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:37.569628 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da25d0-46c2-4b21-948e-b0d5a5397110 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:37.569903 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da25d0-46c2-4b21-948e-b0d5a5397110 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:37.577316 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13da25d0-46c2-4b21-948e-b0d5a5397110 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:37.582828 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13da25d0-46c2-4b21-948e-b0d5a5397110 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:37.590698 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-13da25d0-46c2-4b21-948e-b0d5a5397110 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:32:37.590698 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:32:37.590698 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:32:37.590698 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:32:37.590698 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:32:37.590698 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:32:37.590698 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:32:37.590698 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:32:37.590698 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:32:37.590698 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:32:37.590698 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:32:37.590698 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:32:37.590698 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:32:37.590698 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:32:37.590698 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:32:37.591868 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:32:37.591868 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:32:37.591868 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:32:37.591868 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:32:37.591868 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:32:37.591868 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:32:37.591868 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:32:37.591868 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:32:37.591868 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:32:37.591868 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:32:37.591868 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:32:37.591868 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:32:37.591868 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:32:37.591868 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:32:37.591868 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:32:37.591868 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:32:37.591868 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:32:37.593079 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:32:37.593079 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:32:37.593079 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:32:37.593079 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:32:37.593079 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:32:37.593079 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:32:37.593079 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:32:37.593079 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:32:37.593079 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 612/1244] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:32:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:32:37.598122 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b88d03-ae11-4b21-ae3c-5bb93ae62a7b None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:37.703546 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b88d03-ae11-4b21-ae3c-5bb93ae62a7b None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:37.737903 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32805a2c-1add-4c98-81a4-93780d91231c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:37.738580 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32805a2c-1add-4c98-81a4-93780d91231c None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:37.738867 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32805a2c-1add-4c98-81a4-93780d91231c None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:37.739142 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32805a2c-1add-4c98-81a4-93780d91231c None admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:37.739825 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32805a2c-1add-4c98-81a4-93780d91231c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:32:37.740130 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32805a2c-1add-4c98-81a4-93780d91231c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:37.741585 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32805a2c-1add-4c98-81a4-93780d91231c None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:37.753468 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 633/1245] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Aug 29 22:32:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:32:37.760219 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da25d0-46c2-4b21-948e-b0d5a5397110 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:37.866132 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da25d0-46c2-4b21-948e-b0d5a5397110 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:37.899751 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee220db-a33a-4c29-89e6-3ad9da43e106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:37.900346 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee220db-a33a-4c29-89e6-3ad9da43e106 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:37.900623 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee220db-a33a-4c29-89e6-3ad9da43e106 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:37.901012 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee220db-a33a-4c29-89e6-3ad9da43e106 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:37.901730 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee220db-a33a-4c29-89e6-3ad9da43e106 None admin] RBAC: Adding query filter params (role.id=db83c2f8e21945c3ab9642c620d29a23, scope.project.id=ee84c3340f2641ad963655a7a212ef96, user.id=d8fd0f6e5b144d2ca78b8d96722502a7) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:32:37.902035 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee220db-a33a-4c29-89e6-3ad9da43e106 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:37.903481 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee220db-a33a-4c29-89e6-3ad9da43e106 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:37.909197 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 613/1246] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:32:37 2022] GET /identity/v3/role_assignments?user.id=d8fd0f6e5b144d2ca78b8d96722502a7&scope.project.id=ee84c3340f2641ad963655a7a212ef96&role.id=db83c2f8e21945c3ab9642c620d29a23 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:32:39.977677 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10dcd6-37cf-4ecc-bc31-f31b5d4acf0e None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:39.978234 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10dcd6-37cf-4ecc-bc31-f31b5d4acf0e None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:39.978234 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10dcd6-37cf-4ecc-bc31-f31b5d4acf0e None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:39.979557 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 634/1247] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:32:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:32:39.984646 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95595d22-c6f3-4cfe-a1ad-3340f3a91005 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:39.985115 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95595d22-c6f3-4cfe-a1ad-3340f3a91005 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:39.985223 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95595d22-c6f3-4cfe-a1ad-3340f3a91005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:39.991760 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:40.003773 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-95595d22-c6f3-4cfe-a1ad-3340f3a91005 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:32:40.005183 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-95595d22-c6f3-4cfe-a1ad-3340f3a91005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:32:40.075840 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 614/1248] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:32:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:32:40.349325 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b10dcd6-37cf-4ecc-bc31-f31b5d4acf0e None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:40.370525 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:40.469640 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b10dcd6-37cf-4ecc-bc31-f31b5d4acf0e None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:40.505202 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c10c9-6b13-49c6-9a61-3842c9ce7244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:40.505549 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c10c9-6b13-49c6-9a61-3842c9ce7244 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:40.505728 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c10c9-6b13-49c6-9a61-3842c9ce7244 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:40.505980 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c10c9-6b13-49c6-9a61-3842c9ce7244 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:40.516286 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429c10c9-6b13-49c6-9a61-3842c9ce7244 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:32:40.521135 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-429c10c9-6b13-49c6-9a61-3842c9ce7244 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:32:40.521135 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:32:40.521135 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:32:40.521135 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:32:40.521135 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:32:40.521135 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:32:40.521135 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:32:40.521135 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:32:40.521135 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:32:40.521135 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:32:40.521135 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:32:40.521135 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:32:40.521135 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:32:40.521135 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:32:40.521135 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:32:40.521135 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:32:40.522213 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:32:40.522213 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:32:40.522213 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:32:40.522213 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:32:40.522213 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:32:40.522213 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:32:40.522213 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:32:40.522213 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:32:40.522213 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:32:40.522213 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:32:40.522213 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:32:40.522213 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:32:40.522213 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:32:40.522213 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:32:40.522213 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:32:40.522213 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:32:40.523451 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:32:40.523451 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:32:40.523451 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:32:40.523451 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:32:40.523451 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:32:40.523451 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:32:40.523451 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:32:40.523451 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:32:40.523451 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:32:40.523451 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:32:40.523451 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:32:40.523451 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:32:40.523451 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 635/1249] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Aug 29 22:32:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:32:40.528270 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95595d22-c6f3-4cfe-a1ad-3340f3a91005 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:40.634170 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95595d22-c6f3-4cfe-a1ad-3340f3a91005 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:40.663703 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:40.668518 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ad216-04ac-4520-81cf-8497ac6cc0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:40.669219 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ad216-04ac-4520-81cf-8497ac6cc0b0 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:40.675263 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ad216-04ac-4520-81cf-8497ac6cc0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:40.677096 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ad216-04ac-4520-81cf-8497ac6cc0b0 None admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:40.677800 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3ad216-04ac-4520-81cf-8497ac6cc0b0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:32:40.678382 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3ad216-04ac-4520-81cf-8497ac6cc0b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:40.679885 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3ad216-04ac-4520-81cf-8497ac6cc0b0 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:40.688499 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 615/1250] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:32:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:32:40.694995 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c10c9-6b13-49c6-9a61-3842c9ce7244 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:40.780938 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:40.805707 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c10c9-6b13-49c6-9a61-3842c9ce7244 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:40.839701 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc44ba-7e6b-44b9-a349-da3562653864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:40.840321 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc44ba-7e6b-44b9-a349-da3562653864 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:40.840604 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc44ba-7e6b-44b9-a349-da3562653864 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:40.840914 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc44ba-7e6b-44b9-a349-da3562653864 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:40.851677 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfc44ba-7e6b-44b9-a349-da3562653864 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:40.853165 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfc44ba-7e6b-44b9-a349-da3562653864 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:40.869340 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-8bfc44ba-7e6b-44b9-a349-da3562653864 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:32:40.869340 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:32:40.869340 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:32:40.869340 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:32:40.869340 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:32:40.869340 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:32:40.869340 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:32:40.869340 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:32:40.869340 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:32:40.869340 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:32:40.869340 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:32:40.869340 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:32:40.869340 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:32:40.869340 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:32:40.869340 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:32:40.869340 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:32:40.870626 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:32:40.870626 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:32:40.870626 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:32:40.870626 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:32:40.870626 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:32:40.870626 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:32:40.870626 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:32:40.870626 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:32:40.870626 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:32:40.870626 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:32:40.870626 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:32:40.870626 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:32:40.870626 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:32:40.870626 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:32:40.870626 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:32:40.870626 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:32:40.870626 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:32:40.871558 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:32:40.871558 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:32:40.871558 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:32:40.873974 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 636/1251] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:32:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:32:40.880047 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ad216-04ac-4520-81cf-8497ac6cc0b0 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:40.978802 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ad216-04ac-4520-81cf-8497ac6cc0b0 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:41.013088 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cef185-34de-4d29-990f-834f70b85cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:41.013761 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cef185-34de-4d29-990f-834f70b85cf0 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:41.013949 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cef185-34de-4d29-990f-834f70b85cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:41.014218 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cef185-34de-4d29-990f-834f70b85cf0 None admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:41.014853 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cef185-34de-4d29-990f-834f70b85cf0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:32:41.015151 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cef185-34de-4d29-990f-834f70b85cf0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:41.016513 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cef185-34de-4d29-990f-834f70b85cf0 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:41.029064 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 616/1252] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:32:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:32:41.034907 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc44ba-7e6b-44b9-a349-da3562653864 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:41.144177 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc44ba-7e6b-44b9-a349-da3562653864 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:41.172932 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:41.181886 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f402182-129a-4f17-b151-a5863af94920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:41.182511 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f402182-129a-4f17-b151-a5863af94920 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:41.182789 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f402182-129a-4f17-b151-a5863af94920 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:41.183058 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f402182-129a-4f17-b151-a5863af94920 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:41.190507 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f402182-129a-4f17-b151-a5863af94920 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:41.192581 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f402182-129a-4f17-b151-a5863af94920 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:41.203626 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-9f402182-129a-4f17-b151-a5863af94920 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 22:32:41.203626 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:32:41.203626 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:32:41.203626 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:32:41.203626 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:32:41.203626 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:32:41.203626 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:32:41.203626 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:32:41.203626 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:32:41.203626 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:32:41.203626 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:32:41.203626 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:32:41.203626 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:32:41.203626 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:32:41.203626 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:32:41.203626 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:32:41.204873 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:32:41.204873 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:32:41.204873 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:32:41.204873 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:32:41.204873 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:32:41.204873 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:32:41.204873 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:32:41.204873 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:32:41.204873 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:32:41.204873 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:32:41.204873 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:32:41.204873 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:32:41.204873 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:32:41.204873 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:32:41.204873 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:32:41.204873 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:32:41.204873 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:32:41.204873 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:32:41.206076 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:32:41.206076 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:32:41.206076 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:32:41.206076 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:32:41.206076 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:32:41.206076 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:32:41.206076 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:32:41.206076 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:32:41.206076 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 22:32:41.206076 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:32:41.206076 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 637/1253] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:32:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:32:41.210840 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cef185-34de-4d29-990f-834f70b85cf0 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:41.269628 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:41.315838 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cef185-34de-4d29-990f-834f70b85cf0 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:41.350757 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d866c-92ec-4031-b550-8e5d388a3d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:41.351362 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d866c-92ec-4031-b550-8e5d388a3d9d None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:41.351637 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d866c-92ec-4031-b550-8e5d388a3d9d None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:41.351905 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d866c-92ec-4031-b550-8e5d388a3d9d None admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:41.352622 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8d866c-92ec-4031-b550-8e5d388a3d9d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:32:41.353012 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8d866c-92ec-4031-b550-8e5d388a3d9d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:41.358788 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8d866c-92ec-4031-b550-8e5d388a3d9d None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:41.372423 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 617/1254] 10.222.0.52 () {56 vars in 1143 bytes} [Mon Aug 29 22:32:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:32:41.378498 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f402182-129a-4f17-b151-a5863af94920 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:41.486863 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f402182-129a-4f17-b151-a5863af94920 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:41.524024 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db2bdf-9058-4bf7-82b2-b2a5b2c12cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:41.524540 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db2bdf-9058-4bf7-82b2-b2a5b2c12cfd None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:41.524848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db2bdf-9058-4bf7-82b2-b2a5b2c12cfd None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:41.525120 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db2bdf-9058-4bf7-82b2-b2a5b2c12cfd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:41.583481 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:41.596352 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7db2bdf-9058-4bf7-82b2-b2a5b2c12cfd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:41.598465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7db2bdf-9058-4bf7-82b2-b2a5b2c12cfd None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:41.699306 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 638/1255] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:32:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 22:32:41.705357 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d866c-92ec-4031-b550-8e5d388a3d9d None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:41.810228 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d866c-92ec-4031-b550-8e5d388a3d9d None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:41.824516 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:41.846061 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81f4526-f5fb-4d29-bcf7-62e759698ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:41.846693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f4526-f5fb-4d29-bcf7-62e759698ed5 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:41.846975 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f4526-f5fb-4d29-bcf7-62e759698ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:41.847248 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f4526-f5fb-4d29-bcf7-62e759698ed5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:41.858502 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81f4526-f5fb-4d29-bcf7-62e759698ed5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:41.859988 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81f4526-f5fb-4d29-bcf7-62e759698ed5 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:41.875713 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-a81f4526-f5fb-4d29-bcf7-62e759698ed5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:32:41.875713 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:32:41.875713 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:32:41.875713 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:32:41.875713 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:32:41.875713 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:32:41.875713 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:32:41.875713 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:32:41.875713 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:32:41.875713 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:32:41.875713 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:32:41.875713 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:32:41.875713 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:32:41.875713 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:32:41.875713 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:32:41.875713 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:32:41.882066 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:32:41.882066 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:32:41.882066 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:32:41.882066 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:32:41.882066 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:32:41.882066 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:32:41.882066 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:32:41.882066 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:32:41.882066 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:32:41.882066 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:32:41.882066 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:32:41.882066 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:32:41.882066 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:32:41.882066 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:32:41.882066 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:32:41.882066 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:32:41.882066 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:32:41.883969 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:32:41.883969 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:32:41.883969 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:32:41.883969 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 618/1256] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:32:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:32:41.884197 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db2bdf-9058-4bf7-82b2-b2a5b2c12cfd None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:41.976807 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:41.988629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db2bdf-9058-4bf7-82b2-b2a5b2c12cfd None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:42.023122 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63a4786-18dd-49f3-85c4-9128380671e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:42.024756 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a4786-18dd-49f3-85c4-9128380671e6 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:42.024756 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a4786-18dd-49f3-85c4-9128380671e6 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:42.024756 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a4786-18dd-49f3-85c4-9128380671e6 None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:42.025489 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63a4786-18dd-49f3-85c4-9128380671e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:32:42.025877 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63a4786-18dd-49f3-85c4-9128380671e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:42.027327 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63a4786-18dd-49f3-85c4-9128380671e6 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:42.039618 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 639/1257] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:32:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:32:42.045453 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81f4526-f5fb-4d29-bcf7-62e759698ed5 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:42.149959 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81f4526-f5fb-4d29-bcf7-62e759698ed5 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:42.186427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8900ae-5bbd-4507-aa8b-931604ecd5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:42.187150 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8900ae-5bbd-4507-aa8b-931604ecd5e2 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:42.187431 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8900ae-5bbd-4507-aa8b-931604ecd5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:42.187707 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8900ae-5bbd-4507-aa8b-931604ecd5e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:42.194732 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8900ae-5bbd-4507-aa8b-931604ecd5e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:42.196168 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8900ae-5bbd-4507-aa8b-931604ecd5e2 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:42.208709 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-3b8900ae-5bbd-4507-aa8b-931604ecd5e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:32:42.208709 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:32:42.208709 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:32:42.208709 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:32:42.208709 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:32:42.208709 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:32:42.208709 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:32:42.208709 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:32:42.208709 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:32:42.208709 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:32:42.208709 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:32:42.208709 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:32:42.208709 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:32:42.208709 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:32:42.208709 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:32:42.214744 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:32:42.214744 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:32:42.214744 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:32:42.214744 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:32:42.214744 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:32:42.214744 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:32:42.214744 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:32:42.214744 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:32:42.214744 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:32:42.214744 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:32:42.214744 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:32:42.214744 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:32:42.214744 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:32:42.214744 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:32:42.214744 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:32:42.214744 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:32:42.214744 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:32:42.215869 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:32:42.215869 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:32:42.215869 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:32:42.215869 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:32:42.215869 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:32:42.215869 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:32:42.215869 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:32:42.215869 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:32:42.215869 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 619/1258] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:32:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:32:42.216454 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63a4786-18dd-49f3-85c4-9128380671e6 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:42.320293 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63a4786-18dd-49f3-85c4-9128380671e6 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:42.354664 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c21039-0533-41db-9d08-825c20257454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:42.355255 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c21039-0533-41db-9d08-825c20257454 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:42.355545 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c21039-0533-41db-9d08-825c20257454 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:42.355797 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c21039-0533-41db-9d08-825c20257454 None admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:42.356483 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c21039-0533-41db-9d08-825c20257454 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:32:42.356805 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c21039-0533-41db-9d08-825c20257454 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:42.358218 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c21039-0533-41db-9d08-825c20257454 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:42.370627 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 640/1259] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Aug 29 22:32:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:32:42.376847 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8900ae-5bbd-4507-aa8b-931604ecd5e2 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:42.409828 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:42.488836 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8900ae-5bbd-4507-aa8b-931604ecd5e2 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:42.523140 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8de177-4ba2-4821-be2b-b3e5365d274d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:42.523743 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8de177-4ba2-4821-be2b-b3e5365d274d None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:42.524025 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8de177-4ba2-4821-be2b-b3e5365d274d None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:42.524325 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8de177-4ba2-4821-be2b-b3e5365d274d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:42.525080 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8de177-4ba2-4821-be2b-b3e5365d274d None admin] RBAC: Adding query filter params (role.id=882f35bd847a406eb2222aeafc517a46, scope.project.id=ee84c3340f2641ad963655a7a212ef96, user.id=d8fd0f6e5b144d2ca78b8d96722502a7) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:32:42.525390 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8de177-4ba2-4821-be2b-b3e5365d274d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:42.526873 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8de177-4ba2-4821-be2b-b3e5365d274d None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:42.532373 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 620/1260] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:32:42 2022] GET /identity/v3/role_assignments?user.id=d8fd0f6e5b144d2ca78b8d96722502a7&scope.project.id=ee84c3340f2641ad963655a7a212ef96&role.id=882f35bd847a406eb2222aeafc517a46 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:32:44.443195 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec0f48-61cf-45fc-b1f5-f6e5db83b13c None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:44.443823 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec0f48-61cf-45fc-b1f5-f6e5db83b13c None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:44.443823 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec0f48-61cf-45fc-b1f5-f6e5db83b13c None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:44.445156 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 641/1261] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:32:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:32:44.450250 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812f61d-e829-444b-81d7-86589cbec0c4 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:44.450705 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812f61d-e829-444b-81d7-86589cbec0c4 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:44.450799 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812f61d-e829-444b-81d7-86589cbec0c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:44.471496 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-1812f61d-e829-444b-81d7-86589cbec0c4 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:32:44.472831 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-1812f61d-e829-444b-81d7-86589cbec0c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:32:44.544266 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 621/1262] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:32:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:32:44.816977 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ec0f48-61cf-45fc-b1f5-f6e5db83b13c None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:44.828126 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:44.922384 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ec0f48-61cf-45fc-b1f5-f6e5db83b13c None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:44.958654 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b856f0-ad7f-41f0-8b9a-95dc42231ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:44.959274 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b856f0-ad7f-41f0-8b9a-95dc42231ded None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:44.959547 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b856f0-ad7f-41f0-8b9a-95dc42231ded None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:44.959813 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b856f0-ad7f-41f0-8b9a-95dc42231ded None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:44.973014 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b856f0-ad7f-41f0-8b9a-95dc42231ded None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:32:44.977781 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-b3b856f0-ad7f-41f0-8b9a-95dc42231ded None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:32:44.977781 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:32:44.977781 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:32:44.977781 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:32:44.977781 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:32:44.977781 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:32:44.977781 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:32:44.977781 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:32:44.977781 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:32:44.977781 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:32:44.977781 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:32:44.977781 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:32:44.977781 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:32:44.977781 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:32:44.977781 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:32:44.977781 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:32:44.978920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:32:44.978920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:32:44.978920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:32:44.978920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:32:44.978920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:32:44.978920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:32:44.978920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:32:44.978920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:32:44.978920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:32:44.978920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:32:44.978920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:32:44.978920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:32:44.978920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:32:44.978920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:32:44.978920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:32:44.978920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:32:44.980147 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:32:44.980147 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:32:44.980147 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:32:44.980147 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:32:44.980147 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:32:44.980147 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:32:44.980147 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:32:44.980147 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:32:44.980147 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:32:44.980147 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:32:44.980147 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:32:44.980147 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:32:44.980147 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 642/1263] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Aug 29 22:32:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:32:44.985099 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1812f61d-e829-444b-81d7-86589cbec0c4 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:45.005081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:45.085509 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1812f61d-e829-444b-81d7-86589cbec0c4 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:45.121049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e98fbd-fc4b-4079-a2c1-9cb23d777494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:45.121652 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e98fbd-fc4b-4079-a2c1-9cb23d777494 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:45.121944 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e98fbd-fc4b-4079-a2c1-9cb23d777494 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:45.122308 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e98fbd-fc4b-4079-a2c1-9cb23d777494 None admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:45.122967 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e98fbd-fc4b-4079-a2c1-9cb23d777494 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:32:45.123269 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e98fbd-fc4b-4079-a2c1-9cb23d777494 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:45.124709 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e98fbd-fc4b-4079-a2c1-9cb23d777494 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:45.133639 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 622/1264] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:32:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:32:45.139571 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b856f0-ad7f-41f0-8b9a-95dc42231ded None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:45.208877 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:45.241607 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b856f0-ad7f-41f0-8b9a-95dc42231ded None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:45.275724 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e9cd69-6dca-41e1-b5c4-e3e828a38048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:45.276342 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9cd69-6dca-41e1-b5c4-e3e828a38048 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:45.276618 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9cd69-6dca-41e1-b5c4-e3e828a38048 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:45.276915 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9cd69-6dca-41e1-b5c4-e3e828a38048 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:45.287491 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e9cd69-6dca-41e1-b5c4-e3e828a38048 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:45.288959 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e9cd69-6dca-41e1-b5c4-e3e828a38048 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:45.305086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-69e9cd69-6dca-41e1-b5c4-e3e828a38048 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:32:45.305086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:32:45.305086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:32:45.305086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:32:45.305086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:32:45.305086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:32:45.305086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:32:45.305086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:32:45.305086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:32:45.305086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:32:45.305086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:32:45.305086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:32:45.305086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:32:45.305086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:32:45.305086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:32:45.305086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:32:45.306593 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:32:45.306593 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:32:45.306593 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:32:45.306593 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:32:45.306593 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:32:45.306593 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:32:45.306593 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:32:45.306593 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:32:45.306593 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:32:45.306593 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:32:45.306593 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:32:45.306593 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:32:45.306593 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:32:45.306593 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:32:45.306593 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:32:45.306593 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:32:45.306593 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:32:45.307718 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:32:45.307718 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:32:45.307718 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:32:45.307718 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 643/1265] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:32:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:32:45.311972 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e98fbd-fc4b-4079-a2c1-9cb23d777494 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:45.395638 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:45.414526 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e98fbd-fc4b-4079-a2c1-9cb23d777494 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:45.448363 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e471b7-9ef3-43c0-99ed-6b7c29c7834e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:45.448996 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e471b7-9ef3-43c0-99ed-6b7c29c7834e None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:45.449276 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e471b7-9ef3-43c0-99ed-6b7c29c7834e None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:45.449548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e471b7-9ef3-43c0-99ed-6b7c29c7834e None admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:45.450187 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e471b7-9ef3-43c0-99ed-6b7c29c7834e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:32:45.450514 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e471b7-9ef3-43c0-99ed-6b7c29c7834e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:45.451862 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e471b7-9ef3-43c0-99ed-6b7c29c7834e None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:45.463688 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 623/1266] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:32:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:32:45.469698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e9cd69-6dca-41e1-b5c4-e3e828a38048 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:45.584952 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e9cd69-6dca-41e1-b5c4-e3e828a38048 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:45.604562 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:45.619385 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5585e91c-0b62-4dfd-b149-693b571efd06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:45.620031 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585e91c-0b62-4dfd-b149-693b571efd06 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:45.620312 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585e91c-0b62-4dfd-b149-693b571efd06 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:45.620584 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585e91c-0b62-4dfd-b149-693b571efd06 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:45.628519 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5585e91c-0b62-4dfd-b149-693b571efd06 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:45.634328 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5585e91c-0b62-4dfd-b149-693b571efd06 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:45.641667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-5585e91c-0b62-4dfd-b149-693b571efd06 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 22:32:45.641667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:32:45.641667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:32:45.641667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:32:45.641667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:32:45.641667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:32:45.641667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:32:45.641667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:32:45.641667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:32:45.641667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:32:45.641667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:32:45.641667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:32:45.641667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:32:45.641667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:32:45.641667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:32:45.641667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:32:45.642774 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:32:45.642774 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:32:45.642774 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:32:45.642774 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:32:45.642774 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:32:45.642774 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:32:45.642774 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:32:45.642774 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:32:45.642774 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:32:45.642774 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:32:45.642774 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:32:45.642774 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:32:45.642774 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:32:45.642774 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:32:45.642774 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:32:45.642774 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:32:45.642774 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:32:45.642774 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:32:45.644266 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:32:45.644266 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:32:45.644266 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:32:45.644266 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:32:45.644266 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:32:45.644266 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:32:45.644266 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:32:45.644266 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:32:45.644266 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 22:32:45.644266 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:32:45.644266 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 644/1267] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:32:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:32:45.649454 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e471b7-9ef3-43c0-99ed-6b7c29c7834e None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:45.754684 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e471b7-9ef3-43c0-99ed-6b7c29c7834e None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:45.789553 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a7937-5942-45ef-9f9f-3b96f9ab129d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:45.790145 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a7937-5942-45ef-9f9f-3b96f9ab129d None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:45.790418 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a7937-5942-45ef-9f9f-3b96f9ab129d None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:45.790681 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a7937-5942-45ef-9f9f-3b96f9ab129d None admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:45.791391 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458a7937-5942-45ef-9f9f-3b96f9ab129d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:32:45.791690 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458a7937-5942-45ef-9f9f-3b96f9ab129d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:45.793208 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458a7937-5942-45ef-9f9f-3b96f9ab129d None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:45.802903 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:45.811211 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 624/1268] 10.222.0.52 () {56 vars in 1143 bytes} [Mon Aug 29 22:32:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:32:45.817093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5585e91c-0b62-4dfd-b149-693b571efd06 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:45.924040 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5585e91c-0b62-4dfd-b149-693b571efd06 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:45.960356 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c6531-d6ee-46ef-8db5-6a3db19d30ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:45.961008 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c6531-d6ee-46ef-8db5-6a3db19d30ab None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:45.961283 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c6531-d6ee-46ef-8db5-6a3db19d30ab None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:45.961551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c6531-d6ee-46ef-8db5-6a3db19d30ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:45.972350 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7c6531-d6ee-46ef-8db5-6a3db19d30ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:45.973943 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7c6531-d6ee-46ef-8db5-6a3db19d30ab None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:45.988037 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-fd7c6531-d6ee-46ef-8db5-6a3db19d30ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:32:45.988037 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:32:45.988037 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:32:45.988037 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:32:45.988037 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:32:45.988037 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:32:45.988037 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:32:45.988037 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:32:45.988037 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:32:45.988037 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:32:45.988037 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:32:45.988037 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:32:45.988037 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:32:45.988037 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:32:45.988037 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:32:45.988037 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:32:45.989913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:32:45.989913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:32:45.989913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:32:45.989913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:32:45.989913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:32:45.989913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:32:45.989913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:32:45.989913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:32:45.989913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:32:45.989913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:32:45.989913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:32:45.989913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:32:45.989913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:32:45.989913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:32:45.989913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:32:45.989913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:32:45.989913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:32:45.990938 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:32:45.990938 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:32:45.990938 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:32:45.990938 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 645/1269] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:32:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:32:45.995745 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a7937-5942-45ef-9f9f-3b96f9ab129d None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:46.100842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a7937-5942-45ef-9f9f-3b96f9ab129d None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:46.137595 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd848c0b-ed55-42bc-8735-f89a459bd6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:46.138244 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd848c0b-ed55-42bc-8735-f89a459bd6fa None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:46.143012 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd848c0b-ed55-42bc-8735-f89a459bd6fa None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:46.143012 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd848c0b-ed55-42bc-8735-f89a459bd6fa None admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:46.143012 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd848c0b-ed55-42bc-8735-f89a459bd6fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:32:46.143012 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd848c0b-ed55-42bc-8735-f89a459bd6fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:46.143012 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd848c0b-ed55-42bc-8735-f89a459bd6fa None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:46.152373 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 625/1270] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:32:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:32:46.158527 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c6531-d6ee-46ef-8db5-6a3db19d30ab None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:46.193450 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:46.273167 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c6531-d6ee-46ef-8db5-6a3db19d30ab None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:46.310863 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b1897-5006-46d2-80a5-01fac392631b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:46.311526 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b1897-5006-46d2-80a5-01fac392631b None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:46.311802 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b1897-5006-46d2-80a5-01fac392631b None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:46.312106 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b1897-5006-46d2-80a5-01fac392631b None admin] PATH_INFO: `/v3/projects/service` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:46.319123 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291b1897-5006-46d2-80a5-01fac392631b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:46.320583 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291b1897-5006-46d2-80a5-01fac392631b None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:46.331840 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-291b1897-5006-46d2-80a5-01fac392631b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:32:46.331840 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:32:46.331840 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:32:46.331840 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:32:46.331840 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:32:46.331840 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:32:46.331840 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:32:46.331840 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:32:46.331840 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:32:46.331840 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:32:46.331840 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:32:46.331840 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:32:46.331840 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:32:46.331840 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:32:46.331840 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:32:46.332828 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:32:46.332828 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:32:46.332828 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:32:46.332828 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:32:46.332828 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:32:46.332828 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:32:46.332828 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:32:46.332828 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:32:46.332828 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:32:46.332828 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:32:46.332828 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:32:46.332828 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:32:46.332828 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:32:46.332828 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:32:46.332828 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:32:46.332828 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:32:46.332828 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:32:46.334147 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:32:46.334147 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:32:46.334147 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:32:46.334147 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:32:46.334147 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:32:46.334147 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:32:46.334147 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:32:46.334147 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:32:46.334147 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 646/1271] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:32:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:32:46.339213 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd848c0b-ed55-42bc-8735-f89a459bd6fa None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:46.400857 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:46.443729 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd848c0b-ed55-42bc-8735-f89a459bd6fa None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:46.480422 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e178d3ce-a3dc-42b3-9518-27b79adb37b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:46.481134 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e178d3ce-a3dc-42b3-9518-27b79adb37b2 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:46.481314 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e178d3ce-a3dc-42b3-9518-27b79adb37b2 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:46.481580 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e178d3ce-a3dc-42b3-9518-27b79adb37b2 None admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:46.482259 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e178d3ce-a3dc-42b3-9518-27b79adb37b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:32:46.482558 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e178d3ce-a3dc-42b3-9518-27b79adb37b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:46.483971 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e178d3ce-a3dc-42b3-9518-27b79adb37b2 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:46.498360 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 626/1272] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Aug 29 22:32:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:32:46.506580 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b1897-5006-46d2-80a5-01fac392631b None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:46.608541 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:46.618345 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b1897-5006-46d2-80a5-01fac392631b None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:46.653169 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a07f1a-86d8-4b1d-84e9-670996c42458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:46.653827 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a07f1a-86d8-4b1d-84e9-670996c42458 None admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:46.654107 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a07f1a-86d8-4b1d-84e9-670996c42458 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:46.654382 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a07f1a-86d8-4b1d-84e9-670996c42458 None admin] PATH_INFO: `/v3/projects/ee84c3340f2641ad963655a7a212ef96/users/d8fd0f6e5b144d2ca78b8d96722502a7/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:46.656312 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a07f1a-86d8-4b1d-84e9-670996c42458 None admin] RBAC: Authorizing `identity:create_grant(project_id=ee84c3340f2641ad963655a7a212ef96, user_id=d8fd0f6e5b144d2ca78b8d96722502a7, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:46.657777 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a07f1a-86d8-4b1d-84e9-670996c42458 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:46.755951 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 647/1273] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Aug 29 22:32:46 2022] PUT /identity/v3/projects/ee84c3340f2641ad963655a7a212ef96/users/d8fd0f6e5b144d2ca78b8d96722502a7/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:32:48.672203 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45afd5b6-79e4-43ac-9066-d2977551c6ef None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:48.672843 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45afd5b6-79e4-43ac-9066-d2977551c6ef None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:48.672843 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45afd5b6-79e4-43ac-9066-d2977551c6ef None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:48.674167 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 627/1274] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:32:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:32:48.680059 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bfb52-77a2-4810-9161-96fe2481f095 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:48.680468 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bfb52-77a2-4810-9161-96fe2481f095 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:48.680609 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bfb52-77a2-4810-9161-96fe2481f095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:48.699428 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-fa4bfb52-77a2-4810-9161-96fe2481f095 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:32:48.700616 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-fa4bfb52-77a2-4810-9161-96fe2481f095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:32:48.766358 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 648/1275] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:32:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:32:49.041259 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45afd5b6-79e4-43ac-9066-d2977551c6ef None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:49.146044 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45afd5b6-79e4-43ac-9066-d2977551c6ef None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:49.165275 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:49.188648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd40fd-35f0-4f83-a0a1-c5217256c0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:49.189308 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd40fd-35f0-4f83-a0a1-c5217256c0ec None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:49.198442 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd40fd-35f0-4f83-a0a1-c5217256c0ec None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:49.198442 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd40fd-35f0-4f83-a0a1-c5217256c0ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:49.201762 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bd40fd-35f0-4f83-a0a1-c5217256c0ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:32:49.206712 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-53bd40fd-35f0-4f83-a0a1-c5217256c0ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:32:49.206712 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:32:49.206712 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:32:49.206712 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:32:49.206712 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:32:49.206712 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:32:49.206712 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:32:49.206712 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:32:49.206712 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:32:49.206712 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:32:49.206712 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:32:49.206712 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:32:49.206712 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:32:49.206712 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:32:49.206712 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:32:49.206712 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:32:49.207661 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:32:49.207661 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:32:49.207661 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:32:49.207661 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:32:49.207661 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:32:49.207661 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:32:49.207661 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:32:49.207661 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:32:49.207661 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:32:49.207661 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:32:49.207661 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:32:49.207661 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:32:49.207661 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:32:49.207661 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:32:49.207661 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:32:49.207661 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:32:49.208565 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:32:49.208565 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:32:49.208565 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:32:49.208565 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:32:49.208565 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:32:49.208565 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:32:49.208565 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:32:49.208565 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:32:49.208565 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:32:49.208565 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:32:49.208565 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:32:49.208565 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:32:49.208565 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 628/1276] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Aug 29 22:32:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:32:49.214022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4bfb52-77a2-4810-9161-96fe2481f095 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:49.300193 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:49.319111 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4bfb52-77a2-4810-9161-96fe2481f095 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:49.353197 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a47bea-c157-49b3-af4f-b6a87d84151f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:49.353786 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a47bea-c157-49b3-af4f-b6a87d84151f None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:49.354067 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a47bea-c157-49b3-af4f-b6a87d84151f None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:49.354341 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a47bea-c157-49b3-af4f-b6a87d84151f None admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:49.355000 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a47bea-c157-49b3-af4f-b6a87d84151f None admin] RBAC: Adding query filter params (name=admin) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:32:49.355305 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a47bea-c157-49b3-af4f-b6a87d84151f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:49.356670 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a47bea-c157-49b3-af4f-b6a87d84151f None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:49.365324 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 649/1277] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Aug 29 22:32:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:32:49.371061 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd40fd-35f0-4f83-a0a1-c5217256c0ec None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:49.476828 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd40fd-35f0-4f83-a0a1-c5217256c0ec None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:49.511452 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899634bb-a3e6-4542-8a27-4028c24e7c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:49.512076 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899634bb-a3e6-4542-8a27-4028c24e7c80 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:49.512353 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899634bb-a3e6-4542-8a27-4028c24e7c80 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:49.512621 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899634bb-a3e6-4542-8a27-4028c24e7c80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:49.523288 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899634bb-a3e6-4542-8a27-4028c24e7c80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:49.524892 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899634bb-a3e6-4542-8a27-4028c24e7c80 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:49.542855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-899634bb-a3e6-4542-8a27-4028c24e7c80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:32:49.542855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:32:49.542855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:32:49.542855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:32:49.542855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:32:49.542855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:32:49.542855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:32:49.542855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:32:49.542855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:32:49.542855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:32:49.542855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:32:49.542855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:32:49.542855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:32:49.542855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:32:49.542855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:32:49.542855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:32:49.544052 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:32:49.544052 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:32:49.544052 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:32:49.544052 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:32:49.544052 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:32:49.544052 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:32:49.544052 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:32:49.544052 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:32:49.544052 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:32:49.544052 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:32:49.544052 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:32:49.544052 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:32:49.544052 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:32:49.544052 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:32:49.544052 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:32:49.544052 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:32:49.544052 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:32:49.545183 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:32:49.545183 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:32:49.545183 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:32:49.545183 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 629/1278] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:32:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:32:49.549934 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a47bea-c157-49b3-af4f-b6a87d84151f None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:49.663400 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a47bea-c157-49b3-af4f-b6a87d84151f None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:49.697966 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b77abe-dcf9-4bd3-88e3-43d8ce086d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:49.698649 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b77abe-dcf9-4bd3-88e3-43d8ce086d01 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:49.698831 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b77abe-dcf9-4bd3-88e3-43d8ce086d01 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:49.699108 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b77abe-dcf9-4bd3-88e3-43d8ce086d01 None admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:49.699748 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b77abe-dcf9-4bd3-88e3-43d8ce086d01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:32:49.700050 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b77abe-dcf9-4bd3-88e3-43d8ce086d01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:49.701471 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b77abe-dcf9-4bd3-88e3-43d8ce086d01 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:49.713964 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 650/1279] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:32:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:32:49.719909 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899634bb-a3e6-4542-8a27-4028c24e7c80 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:49.758048 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:49.831555 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899634bb-a3e6-4542-8a27-4028c24e7c80 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:49.866417 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf502b91-8ce3-48a4-a1d9-37c4d3087b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:49.867054 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf502b91-8ce3-48a4-a1d9-37c4d3087b63 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:49.867337 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf502b91-8ce3-48a4-a1d9-37c4d3087b63 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:49.867664 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf502b91-8ce3-48a4-a1d9-37c4d3087b63 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:49.875678 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf502b91-8ce3-48a4-a1d9-37c4d3087b63 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:49.877169 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf502b91-8ce3-48a4-a1d9-37c4d3087b63 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:49.889581 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-bf502b91-8ce3-48a4-a1d9-37c4d3087b63 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 22:32:49.889581 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:32:49.889581 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:32:49.889581 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:32:49.889581 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:32:49.889581 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:32:49.889581 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:32:49.889581 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:32:49.889581 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:32:49.889581 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:32:49.889581 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:32:49.889581 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:32:49.889581 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:32:49.889581 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:32:49.889581 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:32:49.889581 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:32:49.890758 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:32:49.890758 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:32:49.890758 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:32:49.890758 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:32:49.890758 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:32:49.890758 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:32:49.890758 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:32:49.890758 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:32:49.890758 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:32:49.890758 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:32:49.890758 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:32:49.890758 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:32:49.890758 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:32:49.890758 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:32:49.890758 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:32:49.890758 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:32:49.890758 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:32:49.890758 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:32:49.897166 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:32:49.897166 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:32:49.897166 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:32:49.897166 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:32:49.897166 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:32:49.897166 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:32:49.897166 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:32:49.897166 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:32:49.897166 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 22:32:49.897166 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:32:49.897166 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 630/1280] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:32:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:32:49.897927 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b77abe-dcf9-4bd3-88e3-43d8ce086d01 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:49.901677 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:49.998778 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b77abe-dcf9-4bd3-88e3-43d8ce086d01 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:50.035494 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3881721-0eb9-4bb1-8104-5739b98fe96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:50.036337 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3881721-0eb9-4bb1-8104-5739b98fe96c None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:50.036977 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3881721-0eb9-4bb1-8104-5739b98fe96c None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:50.037254 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3881721-0eb9-4bb1-8104-5739b98fe96c None admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:50.037985 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3881721-0eb9-4bb1-8104-5739b98fe96c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:32:50.038291 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3881721-0eb9-4bb1-8104-5739b98fe96c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:50.039713 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3881721-0eb9-4bb1-8104-5739b98fe96c None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:50.058051 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 651/1281] 10.222.0.52 () {56 vars in 1143 bytes} [Mon Aug 29 22:32:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:32:50.064095 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf502b91-8ce3-48a4-a1d9-37c4d3087b63 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:50.159338 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:50.169159 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf502b91-8ce3-48a4-a1d9-37c4d3087b63 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:50.203613 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48abe900-cd0a-43ef-ac01-815d81e1c987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:50.204215 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48abe900-cd0a-43ef-ac01-815d81e1c987 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:50.204492 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48abe900-cd0a-43ef-ac01-815d81e1c987 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:50.204785 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48abe900-cd0a-43ef-ac01-815d81e1c987 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:50.276278 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48abe900-cd0a-43ef-ac01-815d81e1c987 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:50.277797 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48abe900-cd0a-43ef-ac01-815d81e1c987 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:50.370913 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:50.381736 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 631/1282] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Aug 29 22:32:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 22:32:50.387646 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3881721-0eb9-4bb1-8104-5739b98fe96c None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:50.451755 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:50.492065 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3881721-0eb9-4bb1-8104-5739b98fe96c None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:50.526796 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da3eeb-a417-4d18-ab1b-2089e7cca939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:50.527341 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da3eeb-a417-4d18-ab1b-2089e7cca939 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:50.527623 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da3eeb-a417-4d18-ab1b-2089e7cca939 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:50.527898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da3eeb-a417-4d18-ab1b-2089e7cca939 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:50.540534 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2da3eeb-a417-4d18-ab1b-2089e7cca939 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:50.543134 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2da3eeb-a417-4d18-ab1b-2089e7cca939 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:50.563502 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-f2da3eeb-a417-4d18-ab1b-2089e7cca939 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:32:50.563502 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:32:50.563502 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:32:50.563502 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:32:50.563502 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:32:50.563502 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:32:50.563502 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:32:50.563502 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:32:50.563502 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:32:50.563502 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:32:50.563502 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:32:50.563502 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:32:50.563502 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:32:50.563502 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:32:50.563502 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:32:50.563502 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:32:50.564557 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:32:50.564557 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:32:50.564557 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:32:50.564557 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:32:50.564557 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:32:50.564557 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:32:50.564557 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:32:50.564557 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:32:50.564557 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:32:50.564557 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:32:50.564557 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:32:50.564557 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:32:50.564557 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:32:50.564557 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:32:50.564557 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:32:50.564557 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:32:50.564557 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:32:50.565605 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:32:50.565605 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:32:50.565605 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:32:50.565605 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 652/1283] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:32:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:32:50.570581 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48abe900-cd0a-43ef-ac01-815d81e1c987 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:50.674530 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48abe900-cd0a-43ef-ac01-815d81e1c987 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:50.711056 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659d2129-310f-4f22-9824-133590cccbb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:50.711667 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d2129-310f-4f22-9824-133590cccbb9 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:50.711943 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d2129-310f-4f22-9824-133590cccbb9 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:50.712212 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d2129-310f-4f22-9824-133590cccbb9 None admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:50.712896 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659d2129-310f-4f22-9824-133590cccbb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:32:50.713197 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659d2129-310f-4f22-9824-133590cccbb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:50.714771 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659d2129-310f-4f22-9824-133590cccbb9 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:50.727863 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 632/1284] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Aug 29 22:32:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:32:50.733548 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da3eeb-a417-4d18-ab1b-2089e7cca939 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:50.840145 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da3eeb-a417-4d18-ab1b-2089e7cca939 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:50.857220 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:50.878634 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64085961-810f-462d-9c1c-3a6f3741b2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:50.879320 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64085961-810f-462d-9c1c-3a6f3741b2b0 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:50.879690 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64085961-810f-462d-9c1c-3a6f3741b2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:50.879920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64085961-810f-462d-9c1c-3a6f3741b2b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:50.887379 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64085961-810f-462d-9c1c-3a6f3741b2b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:50.888870 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64085961-810f-462d-9c1c-3a6f3741b2b0 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:50.901688 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application [None req-64085961-810f-462d-9c1c-3a6f3741b2b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:32:50.901688 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:32:50.901688 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:32:50.901688 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:32:50.901688 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:32:50.901688 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:32:50.901688 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:32:50.901688 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:32:50.901688 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:32:50.901688 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:32:50.901688 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:32:50.901688 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:32:50.901688 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:32:50.901688 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:32:50.901688 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:32:50.902897 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:32:50.902897 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:32:50.902897 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:32:50.902897 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:32:50.902897 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:32:50.902897 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:32:50.902897 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:32:50.902897 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:32:50.902897 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application with Lock( Aug 29 22:32:50.902897 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:32:50.902897 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:32:50.902897 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:32:50.902897 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:32:50.902897 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:32:50.902897 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:32:50.902897 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:32:50.902897 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:32:50.904088 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:32:50.904088 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:32:50.904088 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:32:50.904088 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:32:50.904088 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:32:50.904088 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:32:50.904088 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:32:50.904088 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: ERROR keystone.server.flask.application Aug 29 22:32:50.904088 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 653/1285] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Aug 29 22:32:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:32:50.908882 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659d2129-310f-4f22-9824-133590cccbb9 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:50.961223 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:51.012356 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659d2129-310f-4f22-9824-133590cccbb9 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:51.047098 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e39104-a691-45ad-8036-4b3389dd0f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:51.055108 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e39104-a691-45ad-8036-4b3389dd0f45 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:51.055108 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e39104-a691-45ad-8036-4b3389dd0f45 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:51.055108 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e39104-a691-45ad-8036-4b3389dd0f45 None admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:51.055108 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e39104-a691-45ad-8036-4b3389dd0f45 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:32:51.055108 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e39104-a691-45ad-8036-4b3389dd0f45 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:51.055108 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e39104-a691-45ad-8036-4b3389dd0f45 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:51.064135 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 633/1286] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Aug 29 22:32:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:32:51.070591 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64085961-810f-462d-9c1c-3a6f3741b2b0 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:51.176245 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64085961-810f-462d-9c1c-3a6f3741b2b0 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:51.213028 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dc1079-0aa5-4237-8f7f-2a9439dd0d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:51.213632 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc1079-0aa5-4237-8f7f-2a9439dd0d8c None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:51.213909 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc1079-0aa5-4237-8f7f-2a9439dd0d8c None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:51.214178 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc1079-0aa5-4237-8f7f-2a9439dd0d8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:51.214895 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dc1079-0aa5-4237-8f7f-2a9439dd0d8c None admin] RBAC: Adding query filter params (role.id=882f35bd847a406eb2222aeafc517a46, scope.project.id=ee84c3340f2641ad963655a7a212ef96, user.id=d8fd0f6e5b144d2ca78b8d96722502a7) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:32:51.215203 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dc1079-0aa5-4237-8f7f-2a9439dd0d8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:51.216698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dc1079-0aa5-4237-8f7f-2a9439dd0d8c None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:51.223038 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 654/1287] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Aug 29 22:32:51 2022] GET /identity/v3/role_assignments?user.id=d8fd0f6e5b144d2ca78b8d96722502a7&scope.project.id=ee84c3340f2641ad963655a7a212ef96&role.id=882f35bd847a406eb2222aeafc517a46 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:32:53.197460 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e53cbb-e47a-4692-a3a5-2f306930e6da None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:53.198401 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e53cbb-e47a-4692-a3a5-2f306930e6da None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:53.198401 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e53cbb-e47a-4692-a3a5-2f306930e6da None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:53.201378 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 634/1288] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:32:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:32:53.205366 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0f70c-757c-42e7-9294-31d913fbb3c4 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:53.205865 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0f70c-757c-42e7-9294-31d913fbb3c4 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:53.205984 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0f70c-757c-42e7-9294-31d913fbb3c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:53.225915 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-27d0f70c-757c-42e7-9294-31d913fbb3c4 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:32:53.227149 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-27d0f70c-757c-42e7-9294-31d913fbb3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:32:53.258711 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:53.299925 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 655/1289] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:32:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:32:53.572021 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e53cbb-e47a-4692-a3a5-2f306930e6da None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:53.678997 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e53cbb-e47a-4692-a3a5-2f306930e6da None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:53.689849 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:53.714188 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b04253e-0a16-463b-8af6-6d88dd5a39ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:53.714825 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b04253e-0a16-463b-8af6-6d88dd5a39ca None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:53.715095 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b04253e-0a16-463b-8af6-6d88dd5a39ca None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:53.715370 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b04253e-0a16-463b-8af6-6d88dd5a39ca None admin] PATH_INFO: `/v3/services/placement` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:53.715950 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b04253e-0a16-463b-8af6-6d88dd5a39ca None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:53.717406 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b04253e-0a16-463b-8af6-6d88dd5a39ca None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:53.726231 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-7b04253e-0a16-463b-8af6-6d88dd5a39ca None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 22:32:53.726231 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:32:53.726231 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:32:53.726231 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:32:53.726231 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:32:53.726231 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:32:53.726231 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:32:53.726231 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:32:53.726231 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 22:32:53.726231 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:32:53.726231 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:32:53.726231 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:32:53.726231 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:32:53.726231 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:32:53.726231 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:32:53.726231 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:32:53.726231 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:32:53.727520 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:32:53.727520 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:32:53.727520 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:32:53.727520 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:32:53.727520 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:32:53.727520 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:32:53.727520 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:32:53.727520 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:32:53.727520 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:32:53.727520 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:32:53.727520 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:32:53.727520 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:32:53.727520 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:32:53.727520 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:32:53.727520 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:32:53.727520 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:32:53.728387 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:32:53.728387 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:32:53.728387 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:32:53.728387 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:32:53.728387 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:32:53.728387 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:32:53.728387 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:32:53.728387 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:32:53.728387 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:32:53.728387 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:32:53.728387 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:32:53.728387 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:32:53.728387 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 22:32:53.728387 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:32:53.728387 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 635/1290] 10.222.0.52 () {56 vars in 1117 bytes} [Mon Aug 29 22:32:53 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:32:53.733421 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0f70c-757c-42e7-9294-31d913fbb3c4 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:53.837592 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0f70c-757c-42e7-9294-31d913fbb3c4 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:53.872172 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc0a78b-9aac-40f9-ab80-ce77b4b49b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:53.872866 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0a78b-9aac-40f9-ab80-ce77b4b49b24 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:53.873148 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0a78b-9aac-40f9-ab80-ce77b4b49b24 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:53.873418 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0a78b-9aac-40f9-ab80-ce77b4b49b24 None admin] PATH_INFO: `/v3/services` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:53.874036 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc0a78b-9aac-40f9-ab80-ce77b4b49b24 None admin] RBAC: Adding query filter params (name=placement) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:32:53.874332 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc0a78b-9aac-40f9-ab80-ce77b4b49b24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:53.875675 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc0a78b-9aac-40f9-ab80-ce77b4b49b24 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:53.881589 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 656/1291] 10.222.0.52 () {56 vars in 1116 bytes} [Mon Aug 29 22:32:53 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:32:53.887436 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b04253e-0a16-463b-8af6-6d88dd5a39ca None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:53.993385 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b04253e-0a16-463b-8af6-6d88dd5a39ca None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:54.026672 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1709402-842b-43fe-ab7e-8ee3d940d3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:54.027374 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1709402-842b-43fe-ab7e-8ee3d940d3f1 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:54.027529 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1709402-842b-43fe-ab7e-8ee3d940d3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:54.027802 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1709402-842b-43fe-ab7e-8ee3d940d3f1 None admin] PATH_INFO: `/v3/services` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:54.028424 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1709402-842b-43fe-ab7e-8ee3d940d3f1 None admin] RBAC: Adding query filter params (type=placement) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:32:54.028752 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1709402-842b-43fe-ab7e-8ee3d940d3f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:54.030110 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1709402-842b-43fe-ab7e-8ee3d940d3f1 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:54.034478 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 636/1292] 10.222.0.52 () {56 vars in 1116 bytes} [Mon Aug 29 22:32:53 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:32:55.965696 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5249b482-3d7b-41ef-bd6b-0910adee0121 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:55.966226 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5249b482-3d7b-41ef-bd6b-0910adee0121 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:55.966342 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5249b482-3d7b-41ef-bd6b-0910adee0121 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:55.967591 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 657/1293] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:32:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:32:55.972933 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92c6a0-8da3-44b9-99ff-27fc9fd4c6a6 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:55.973961 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92c6a0-8da3-44b9-99ff-27fc9fd4c6a6 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:55.974211 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92c6a0-8da3-44b9-99ff-27fc9fd4c6a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:55.994492 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-1f92c6a0-8da3-44b9-99ff-27fc9fd4c6a6 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:32:55.995696 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-1f92c6a0-8da3-44b9-99ff-27fc9fd4c6a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:32:56.032783 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:56.072863 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 637/1294] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:32:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:32:56.349075 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5249b482-3d7b-41ef-bd6b-0910adee0121 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:56.384035 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:56.460760 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5249b482-3d7b-41ef-bd6b-0910adee0121 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:56.495822 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d994f742-f113-4b7f-9adc-439d528de621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:56.496477 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d994f742-f113-4b7f-9adc-439d528de621 None admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:56.496789 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d994f742-f113-4b7f-9adc-439d528de621 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:56.497065 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d994f742-f113-4b7f-9adc-439d528de621 None admin] PATH_INFO: `/v3/services` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:56.497750 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d994f742-f113-4b7f-9adc-439d528de621 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:56.499155 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d994f742-f113-4b7f-9adc-439d528de621 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:56.593606 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 658/1295] 10.222.0.52 () {60 vars in 1141 bytes} [Mon Aug 29 22:32:56 2022] POST /identity/v3/services => generated 246 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:32:58.595662 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd9776b-ac98-45bc-8716-903669c3ef2c None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:58.596170 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd9776b-ac98-45bc-8716-903669c3ef2c None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:58.596260 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd9776b-ac98-45bc-8716-903669c3ef2c None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:58.597600 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 638/1296] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:32:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:32:58.603595 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23dc9a3-dc2b-4bcc-8f64-209d63061b1d None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:58.604015 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23dc9a3-dc2b-4bcc-8f64-209d63061b1d None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:58.604146 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23dc9a3-dc2b-4bcc-8f64-209d63061b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:58.622078 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-a23dc9a3-dc2b-4bcc-8f64-209d63061b1d None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:32:58.623241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-a23dc9a3-dc2b-4bcc-8f64-209d63061b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:32:58.696789 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 659/1297] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:32:58 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:32:58.967343 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd9776b-ac98-45bc-8716-903669c3ef2c None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:59.073305 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd9776b-ac98-45bc-8716-903669c3ef2c None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:59.107789 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f8405-c048-4f42-a2a0-eec6916ec3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:59.108398 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f8405-c048-4f42-a2a0-eec6916ec3a9 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:59.108669 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f8405-c048-4f42-a2a0-eec6916ec3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:59.108965 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f8405-c048-4f42-a2a0-eec6916ec3a9 None admin] PATH_INFO: `/v3/services/placement` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:59.109535 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6f8405-c048-4f42-a2a0-eec6916ec3a9 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:59.110908 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6f8405-c048-4f42-a2a0-eec6916ec3a9 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:59.120258 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-4d6f8405-c048-4f42-a2a0-eec6916ec3a9 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 22:32:59.120258 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:32:59.120258 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:32:59.120258 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:32:59.120258 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:32:59.120258 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:32:59.120258 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:32:59.120258 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:32:59.120258 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 22:32:59.120258 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:32:59.120258 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:32:59.120258 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:32:59.120258 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:32:59.120258 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:32:59.120258 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:32:59.120258 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:32:59.120258 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:32:59.121479 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:32:59.121479 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:32:59.121479 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:32:59.121479 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:32:59.121479 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:32:59.121479 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:32:59.121479 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:32:59.121479 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:32:59.121479 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:32:59.121479 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:32:59.121479 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:32:59.121479 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:32:59.121479 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:32:59.121479 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:32:59.121479 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:32:59.121479 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:32:59.122522 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:32:59.122522 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:32:59.122522 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:32:59.122522 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:32:59.122522 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:32:59.122522 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:32:59.122522 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:32:59.122522 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:32:59.122522 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:32:59.122522 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:32:59.122522 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:32:59.122522 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:32:59.122522 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 22:32:59.122522 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:32:59.122522 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 639/1298] 10.222.0.52 () {56 vars in 1117 bytes} [Mon Aug 29 22:32:58 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:32:59.127410 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23dc9a3-dc2b-4bcc-8f64-209d63061b1d None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:59.146101 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:59.229218 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23dc9a3-dc2b-4bcc-8f64-209d63061b1d None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:59.263650 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab7da77-e75b-406b-b089-50adeda29061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:59.264253 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7da77-e75b-406b-b089-50adeda29061 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:59.264535 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7da77-e75b-406b-b089-50adeda29061 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:59.264841 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7da77-e75b-406b-b089-50adeda29061 None admin] PATH_INFO: `/v3/services` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:59.265467 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab7da77-e75b-406b-b089-50adeda29061 None admin] RBAC: Adding query filter params (name=placement) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:32:59.265769 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab7da77-e75b-406b-b089-50adeda29061 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:59.267128 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab7da77-e75b-406b-b089-50adeda29061 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:59.272853 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 660/1299] 10.222.0.52 () {56 vars in 1116 bytes} [Mon Aug 29 22:32:59 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:32:59.278926 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f8405-c048-4f42-a2a0-eec6916ec3a9 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:59.311218 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:59.382863 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f8405-c048-4f42-a2a0-eec6916ec3a9 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:59.418897 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc52732-71e3-4b7c-88a8-efadea8b4919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:59.419504 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc52732-71e3-4b7c-88a8-efadea8b4919 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:59.419784 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc52732-71e3-4b7c-88a8-efadea8b4919 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:59.420076 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc52732-71e3-4b7c-88a8-efadea8b4919 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:59.420797 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc52732-71e3-4b7c-88a8-efadea8b4919 None admin] RBAC: Adding query filter params (interface=public, service_id=dde68c3f94354bd69f41fcbb93712507, region_id=RegionOne) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:32:59.421104 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc52732-71e3-4b7c-88a8-efadea8b4919 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:59.422458 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc52732-71e3-4b7c-88a8-efadea8b4919 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:59.428056 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 640/1300] 10.222.0.52 () {56 vars in 1251 bytes} [Mon Aug 29 22:32:59 2022] GET /identity/v3/endpoints?service_id=dde68c3f94354bd69f41fcbb93712507&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:32:59.434183 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab7da77-e75b-406b-b089-50adeda29061 None admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:32:59.517143 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:32:59.536028 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab7da77-e75b-406b-b089-50adeda29061 None admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:32:59.571415 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50ca36c-a98f-4630-bbf0-aadb22c474b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:32:59.572008 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ca36c-a98f-4630-bbf0-aadb22c474b0 None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:32:59.572290 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ca36c-a98f-4630-bbf0-aadb22c474b0 None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:32:59.572566 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ca36c-a98f-4630-bbf0-aadb22c474b0 None admin] PATH_INFO: `/v3/services` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:32:59.573195 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50ca36c-a98f-4630-bbf0-aadb22c474b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:32:59.574551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50ca36c-a98f-4630-bbf0-aadb22c474b0 None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:32:59.581304 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 661/1301] 10.222.0.52 () {56 vars in 1087 bytes} [Mon Aug 29 22:32:59 2022] GET /identity/v3/services => generated 2191 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:33:01.489396 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e311e-f87f-4281-af0f-eb3a42175598 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:33:01.490037 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e311e-f87f-4281-af0f-eb3a42175598 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:33:01.490037 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e311e-f87f-4281-af0f-eb3a42175598 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:33:01.491255 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 641/1302] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:33:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:33:01.496311 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010293f-eb4f-44d3-a7b9-fd8d1ddf5754 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:33:01.496776 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010293f-eb4f-44d3-a7b9-fd8d1ddf5754 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:33:01.496895 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010293f-eb4f-44d3-a7b9-fd8d1ddf5754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:33:01.515467 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-4010293f-eb4f-44d3-a7b9-fd8d1ddf5754 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:33:01.517168 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-4010293f-eb4f-44d3-a7b9-fd8d1ddf5754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:33:01.585235 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 662/1303] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:33:01 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:33:01.861946 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e311e-f87f-4281-af0f-eb3a42175598 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:33:01.959659 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:33:01.966221 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e311e-f87f-4281-af0f-eb3a42175598 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:33:02.001108 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed61004-a6c7-4467-8f27-ffe4717d70b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:33:02.001758 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed61004-a6c7-4467-8f27-ffe4717d70b2 None admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:33:02.002012 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed61004-a6c7-4467-8f27-ffe4717d70b2 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:33:02.002282 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed61004-a6c7-4467-8f27-ffe4717d70b2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:33:02.002855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed61004-a6c7-4467-8f27-ffe4717d70b2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:33:02.004240 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed61004-a6c7-4467-8f27-ffe4717d70b2 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:33:02.012911 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application [None req-fed61004-a6c7-4467-8f27-ffe4717d70b2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 22:33:02.012911 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:33:02.012911 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:33:02.012911 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:33:02.012911 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:33:02.012911 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:33:02.012911 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:33:02.012911 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:33:02.012911 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 22:33:02.012911 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:33:02.012911 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:33:02.012911 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:33:02.012911 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:33:02.012911 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:33:02.012911 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:33:02.012911 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:33:02.012911 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:33:02.014089 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:33:02.014089 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:33:02.014089 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:33:02.014089 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:33:02.014089 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:33:02.014089 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:33:02.014089 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:33:02.014089 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:33:02.014089 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:33:02.014089 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:33:02.014089 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:33:02.014089 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:33:02.014089 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:33:02.014089 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:33:02.014089 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:33:02.014089 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:33:02.015100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:33:02.015100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application with Lock( Aug 29 22:33:02.015100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:33:02.015100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:33:02.015100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:33:02.015100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:33:02.015100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:33:02.015100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:33:02.015100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:33:02.015100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:33:02.015100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:33:02.015100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:33:02.015100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 22:33:02.015100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: ERROR keystone.server.flask.application Aug 29 22:33:02.015100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 642/1304] 10.222.0.52 () {56 vars in 1117 bytes} [Mon Aug 29 22:33:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:33:02.020138 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4010293f-eb4f-44d3-a7b9-fd8d1ddf5754 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:33:02.104610 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:33:02.127804 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4010293f-eb4f-44d3-a7b9-fd8d1ddf5754 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:33:02.162538 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b19ede5-00ed-40f6-a538-bad376c8bd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:33:02.163146 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19ede5-00ed-40f6-a538-bad376c8bd8b None admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:33:02.163428 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19ede5-00ed-40f6-a538-bad376c8bd8b None admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:33:02.163701 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19ede5-00ed-40f6-a538-bad376c8bd8b None admin] PATH_INFO: `/v3/services` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:33:02.164320 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b19ede5-00ed-40f6-a538-bad376c8bd8b None admin] RBAC: Adding query filter params (name=placement) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:33:02.164623 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b19ede5-00ed-40f6-a538-bad376c8bd8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:33:02.166024 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b19ede5-00ed-40f6-a538-bad376c8bd8b None admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:33:02.171187 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 663/1305] 10.222.0.52 () {56 vars in 1116 bytes} [Mon Aug 29 22:33:02 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:33:02.177268 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed61004-a6c7-4467-8f27-ffe4717d70b2 None admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:33:02.275923 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed61004-a6c7-4467-8f27-ffe4717d70b2 None admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:33:02.310766 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc767bd-370f-4b04-9a40-0c8f646028b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:33:02.311361 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc767bd-370f-4b04-9a40-0c8f646028b7 None admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:33:02.311635 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc767bd-370f-4b04-9a40-0c8f646028b7 None admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:33:02.311900 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc767bd-370f-4b04-9a40-0c8f646028b7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:33:02.312564 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc767bd-370f-4b04-9a40-0c8f646028b7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:33:02.313942 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc767bd-370f-4b04-9a40-0c8f646028b7 None admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:33:02.418434 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 643/1306] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:33:02 2022] POST /identity/v3/endpoints => generated 331 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:33:12.192632 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=30 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:33:12.414011 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:39:55.524761 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f53eb2-d002-4d5a-85a3-45e8f236455f None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:39:55.524761 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f53eb2-d002-4d5a-85a3-45e8f236455f None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:39:55.525512 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f53eb2-d002-4d5a-85a3-45e8f236455f None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:39:55.532025 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4386a1-60c1-498d-b9f7-19d55b0294fb None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:39:55.532542 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4386a1-60c1-498d-b9f7-19d55b0294fb None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:39:55.532636 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4386a1-60c1-498d-b9f7-19d55b0294fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:39:55.537721 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 664/1307] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:39:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:39:55.621088 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-cb4386a1-60c1-498d-b9f7-19d55b0294fb None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:39:55.622401 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-cb4386a1-60c1-498d-b9f7-19d55b0294fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:39:55.746716 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 644/1308] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:39:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:39:56.053469 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079140cb-cfbf-4800-ab68-6b41bd204425 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:39:56.053669 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079140cb-cfbf-4800-ab68-6b41bd204425 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:39:56.053926 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079140cb-cfbf-4800-ab68-6b41bd204425 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:39:56.055792 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 665/1309] 10.222.0.52 () {54 vars in 965 bytes} [Mon Aug 29 22:39:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:39:56.062560 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ebf38c-bf40-497b-b038-89f05480acb2 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:39:56.063146 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ebf38c-bf40-497b-b038-89f05480acb2 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:39:56.063439 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ebf38c-bf40-497b-b038-89f05480acb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:39:56.162195 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-a2ebf38c-bf40-497b-b038-89f05480acb2 None None] MFA Rules not processed for user `0fe5e8c4cb324ffb9d5b80f18d123727`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:39:56.163466 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-a2ebf38c-bf40-497b-b038-89f05480acb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:39:56.189043 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:39:56.264884 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 645/1310] 10.222.0.52 () {58 vars in 1062 bytes} [Mon Aug 29 22:39:56 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:39:56.271458 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079140cb-cfbf-4800-ab68-6b41bd204425 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:39:56.273444 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-079140cb-cfbf-4800-ab68-6b41bd204425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:39:56.394997 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079140cb-cfbf-4800-ab68-6b41bd204425 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:39:56.476763 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f735da02-032d-4a3e-9187-d9437c6c480b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:39:56.477254 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f735da02-032d-4a3e-9187-d9437c6c480b service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:39:56.477547 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f735da02-032d-4a3e-9187-d9437c6c480b service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:39:56.477826 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f735da02-032d-4a3e-9187-d9437c6c480b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:39:56.479425 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-f735da02-032d-4a3e-9187-d9437c6c480b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:39:56.542883 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:39:56.604979 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f735da02-032d-4a3e-9187-d9437c6c480b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:39:56.606355 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f735da02-032d-4a3e-9187-d9437c6c480b service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:39:56.627395 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 666/1311] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:39:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:39:58.873056 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d0c18-7e5b-4cd0-a7b9-8cecf84e650b None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:39:58.873533 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d0c18-7e5b-4cd0-a7b9-8cecf84e650b None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:39:58.873764 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d0c18-7e5b-4cd0-a7b9-8cecf84e650b None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:39:58.875534 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 646/1312] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:39:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:39:58.881579 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a925135-56ce-4e2b-847e-bb547d0922ab None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:39:58.882025 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a925135-56ce-4e2b-847e-bb547d0922ab None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:39:58.882662 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a925135-56ce-4e2b-847e-bb547d0922ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:39:58.984804 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-3a925135-56ce-4e2b-847e-bb547d0922ab None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:39:58.986006 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-3a925135-56ce-4e2b-847e-bb547d0922ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:39:59.083871 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 667/1313] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:39:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:39:59.365823 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d0c18-7e5b-4cd0-a7b9-8cecf84e650b None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:39:59.368397 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-7f7d0c18-7e5b-4cd0-a7b9-8cecf84e650b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:39:59.477218 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d0c18-7e5b-4cd0-a7b9-8cecf84e650b None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:39:59.553598 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:39:59.556825 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e221ae-682a-42ce-9df3-3d6faba65d8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:39:59.557436 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e221ae-682a-42ce-9df3-3d6faba65d8f service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:39:59.557714 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e221ae-682a-42ce-9df3-3d6faba65d8f service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:39:59.557984 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e221ae-682a-42ce-9df3-3d6faba65d8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:39:59.559578 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-95e221ae-682a-42ce-9df3-3d6faba65d8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:39:59.680169 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e221ae-682a-42ce-9df3-3d6faba65d8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:39:59.681563 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e221ae-682a-42ce-9df3-3d6faba65d8f service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:39:59.701290 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 647/1314] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:39:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:40:02.178604 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7fdc5-3d31-428d-9791-ea37fd36a019 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:02.179225 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7fdc5-3d31-428d-9791-ea37fd36a019 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:02.179225 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7fdc5-3d31-428d-9791-ea37fd36a019 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:02.181494 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 668/1315] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:40:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:40:02.187880 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868ecf1-634e-495b-a341-d744b4acfd04 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:02.188616 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868ecf1-634e-495b-a341-d744b4acfd04 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:02.188740 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868ecf1-634e-495b-a341-d744b4acfd04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:02.257411 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-8868ecf1-634e-495b-a341-d744b4acfd04 None None] MFA Rules not processed for user `3a00a8e520484973b5ffa1769dcee1fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:40:02.258591 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-8868ecf1-634e-495b-a341-d744b4acfd04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:02.309826 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:40:02.354318 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 648/1316] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:40:02 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:40:02.636118 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe7fdc5-3d31-428d-9791-ea37fd36a019 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:40:02.637695 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-abe7fdc5-3d31-428d-9791-ea37fd36a019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:02.647594 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:40:02.743328 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe7fdc5-3d31-428d-9791-ea37fd36a019 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:40:02.816743 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82460d0c-dd0f-4dd2-ad52-517d8ea431a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:40:02.817369 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82460d0c-dd0f-4dd2-ad52-517d8ea431a6 service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:02.817651 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82460d0c-dd0f-4dd2-ad52-517d8ea431a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:02.817922 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82460d0c-dd0f-4dd2-ad52-517d8ea431a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:02.819526 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-82460d0c-dd0f-4dd2-ad52-517d8ea431a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:02.881557 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:40:02.931251 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82460d0c-dd0f-4dd2-ad52-517d8ea431a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:40:02.933060 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82460d0c-dd0f-4dd2-ad52-517d8ea431a6 service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:40:02.955709 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 669/1317] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:40:02 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:40:05.721544 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67845637-232f-4c63-958c-62f57f09fee7 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:05.722098 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67845637-232f-4c63-958c-62f57f09fee7 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:05.722098 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67845637-232f-4c63-958c-62f57f09fee7 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:05.723430 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 649/1318] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:40:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:40:05.730136 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4921f65-2a2e-4d6d-910f-3b5f7d8e3a4b None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:05.730549 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4921f65-2a2e-4d6d-910f-3b5f7d8e3a4b None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:05.730687 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4921f65-2a2e-4d6d-910f-3b5f7d8e3a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:05.796946 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-a4921f65-2a2e-4d6d-910f-3b5f7d8e3a4b None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:40:05.798077 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-a4921f65-2a2e-4d6d-910f-3b5f7d8e3a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:05.896909 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 670/1319] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:40:05 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:40:06.180646 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67845637-232f-4c63-958c-62f57f09fee7 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:40:06.182328 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-67845637-232f-4c63-958c-62f57f09fee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:06.312185 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67845637-232f-4c63-958c-62f57f09fee7 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:40:06.390788 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0aeccf-e452-4bac-a723-c5ec83e71c31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:40:06.391390 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0aeccf-e452-4bac-a723-c5ec83e71c31 service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:06.391666 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0aeccf-e452-4bac-a723-c5ec83e71c31 service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:06.391935 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0aeccf-e452-4bac-a723-c5ec83e71c31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:06.393568 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-fd0aeccf-e452-4bac-a723-c5ec83e71c31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:06.401790 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:40:06.502491 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0aeccf-e452-4bac-a723-c5ec83e71c31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:40:06.503929 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0aeccf-e452-4bac-a723-c5ec83e71c31 service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:40:06.522636 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 650/1320] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:40:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:40:09.556874 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e3fee9-b2e7-4700-826c-03d6ca36cfb0 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:09.557419 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e3fee9-b2e7-4700-826c-03d6ca36cfb0 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:09.557419 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e3fee9-b2e7-4700-826c-03d6ca36cfb0 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:09.558761 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 671/1321] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:40:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:40:09.564068 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf3a82-f7bd-4aba-9a85-8cef5fe085c1 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:09.564459 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf3a82-f7bd-4aba-9a85-8cef5fe085c1 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:09.564612 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf3a82-f7bd-4aba-9a85-8cef5fe085c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:09.630168 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-dfcf3a82-f7bd-4aba-9a85-8cef5fe085c1 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:40:09.631359 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-dfcf3a82-f7bd-4aba-9a85-8cef5fe085c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:09.693551 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:40:09.734454 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 651/1322] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:40:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:40:10.016290 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e3fee9-b2e7-4700-826c-03d6ca36cfb0 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:40:10.017963 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-80e3fee9-b2e7-4700-826c-03d6ca36cfb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:10.042428 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:40:10.137209 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e3fee9-b2e7-4700-826c-03d6ca36cfb0 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:40:10.218840 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767694e5-2543-445c-a52a-635d8939b1e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:40:10.219453 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767694e5-2543-445c-a52a-635d8939b1e3 service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:10.219732 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767694e5-2543-445c-a52a-635d8939b1e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:10.220004 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767694e5-2543-445c-a52a-635d8939b1e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:10.221648 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-767694e5-2543-445c-a52a-635d8939b1e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:10.295936 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:40:10.341781 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767694e5-2543-445c-a52a-635d8939b1e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:40:10.343214 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767694e5-2543-445c-a52a-635d8939b1e3 service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:40:10.364179 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 672/1323] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:40:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:40:12.424034 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf01b8-8f93-4e03-9e8d-f7ce6f7dd164 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:12.424553 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf01b8-8f93-4e03-9e8d-f7ce6f7dd164 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:12.424553 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf01b8-8f93-4e03-9e8d-f7ce6f7dd164 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:12.426066 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 652/1324] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:40:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:40:12.431483 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45ba00-f4c9-4127-a552-e7dddb65e2f4 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:12.431979 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45ba00-f4c9-4127-a552-e7dddb65e2f4 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:12.432083 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45ba00-f4c9-4127-a552-e7dddb65e2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:12.508415 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-da45ba00-f4c9-4127-a552-e7dddb65e2f4 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:40:12.509981 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-da45ba00-f4c9-4127-a552-e7dddb65e2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:12.623399 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 673/1325] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:40:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:40:12.910461 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf01b8-8f93-4e03-9e8d-f7ce6f7dd164 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:40:12.912159 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-23bf01b8-8f93-4e03-9e8d-f7ce6f7dd164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:13.034029 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf01b8-8f93-4e03-9e8d-f7ce6f7dd164 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:40:13.136420 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e0ee99-61e1-4643-8c94-472952ee48e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:40:13.137072 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e0ee99-61e1-4643-8c94-472952ee48e5 service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:13.137350 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e0ee99-61e1-4643-8c94-472952ee48e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:13.137630 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e0ee99-61e1-4643-8c94-472952ee48e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:13.139338 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-28e0ee99-61e1-4643-8c94-472952ee48e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:13.161540 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:40:13.267743 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e0ee99-61e1-4643-8c94-472952ee48e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:40:13.269231 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e0ee99-61e1-4643-8c94-472952ee48e5 service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:40:13.289094 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 653/1326] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:40:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:40:15.301956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bcc845-d9b2-4770-803a-13cba0032266 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:15.304228 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bcc845-d9b2-4770-803a-13cba0032266 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:15.304228 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bcc845-d9b2-4770-803a-13cba0032266 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:15.304228 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 674/1327] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:40:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:40:15.310117 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca9e468-13b2-450a-8af9-75b0c557eaf6 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:15.310607 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca9e468-13b2-450a-8af9-75b0c557eaf6 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:15.310841 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca9e468-13b2-450a-8af9-75b0c557eaf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:15.386268 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-9ca9e468-13b2-450a-8af9-75b0c557eaf6 None None] MFA Rules not processed for user `3a00a8e520484973b5ffa1769dcee1fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:40:15.387498 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-9ca9e468-13b2-450a-8af9-75b0c557eaf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:15.456329 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:40:15.488639 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 654/1328] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:40:15 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:40:15.804482 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c9e4e-f870-4988-b28b-f762eca747d6 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:15.805022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c9e4e-f870-4988-b28b-f762eca747d6 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:15.805297 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c9e4e-f870-4988-b28b-f762eca747d6 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:15.806632 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 675/1329] 10.222.0.52 () {54 vars in 965 bytes} [Mon Aug 29 22:40:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:40:15.812496 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57bd565-5a9a-4e48-930e-a23df3ed8ef5 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:15.812856 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57bd565-5a9a-4e48-930e-a23df3ed8ef5 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:15.813132 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57bd565-5a9a-4e48-930e-a23df3ed8ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:15.904906 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-c57bd565-5a9a-4e48-930e-a23df3ed8ef5 None None] MFA Rules not processed for user `0fe5e8c4cb324ffb9d5b80f18d123727`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:40:15.906110 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-c57bd565-5a9a-4e48-930e-a23df3ed8ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:15.985373 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 655/1330] 10.222.0.52 () {58 vars in 1062 bytes} [Mon Aug 29 22:40:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:40:15.994397 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c9e4e-f870-4988-b28b-f762eca747d6 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:40:15.995949 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-651c9e4e-f870-4988-b28b-f762eca747d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:16.028058 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:40:16.108097 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c9e4e-f870-4988-b28b-f762eca747d6 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:40:16.183634 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2186e9-25b9-401d-beda-5a97e7c29936 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:40:16.184153 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2186e9-25b9-401d-beda-5a97e7c29936 service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:16.184429 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2186e9-25b9-401d-beda-5a97e7c29936 service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:16.184697 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2186e9-25b9-401d-beda-5a97e7c29936 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:16.186358 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-8d2186e9-25b9-401d-beda-5a97e7c29936 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:16.265127 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:40:16.297922 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2186e9-25b9-401d-beda-5a97e7c29936 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:40:16.299452 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2186e9-25b9-401d-beda-5a97e7c29936 service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:40:16.319493 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 676/1331] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:40:15 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:40:19.089732 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5247a0-092b-46d6-8322-a87850ad0766 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:19.090246 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5247a0-092b-46d6-8322-a87850ad0766 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:19.090357 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5247a0-092b-46d6-8322-a87850ad0766 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:19.091596 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 656/1332] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:40:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:40:19.096903 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b27d6-50ed-4262-8d91-b6ee732eb93b None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:19.097307 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b27d6-50ed-4262-8d91-b6ee732eb93b None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:19.097456 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b27d6-50ed-4262-8d91-b6ee732eb93b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:19.181029 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-be8b27d6-50ed-4262-8d91-b6ee732eb93b None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:40:19.182151 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-be8b27d6-50ed-4262-8d91-b6ee732eb93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:19.283744 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 677/1333] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:40:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:40:19.584842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5247a0-092b-46d6-8322-a87850ad0766 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:40:19.586491 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-ac5247a0-092b-46d6-8322-a87850ad0766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:19.656648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:40:19.692025 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5247a0-092b-46d6-8322-a87850ad0766 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:40:19.766038 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ce6f78-7507-41f2-9c3c-caeb35576be6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:40:19.766636 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce6f78-7507-41f2-9c3c-caeb35576be6 service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:19.766915 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce6f78-7507-41f2-9c3c-caeb35576be6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:19.767186 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce6f78-7507-41f2-9c3c-caeb35576be6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:19.768792 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-45ce6f78-7507-41f2-9c3c-caeb35576be6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:19.882902 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ce6f78-7507-41f2-9c3c-caeb35576be6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:40:19.884250 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ce6f78-7507-41f2-9c3c-caeb35576be6 service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:40:19.894054 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:40:19.902116 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 657/1334] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:40:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:40:21.881826 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b89b7-40d2-4067-aa7d-aef669401f12 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:21.884710 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b89b7-40d2-4067-aa7d-aef669401f12 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:21.884710 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b89b7-40d2-4067-aa7d-aef669401f12 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:21.884710 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 678/1335] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:40:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:40:21.891325 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac9921-65ec-481c-8a0e-54811ea3f2b3 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:21.891776 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac9921-65ec-481c-8a0e-54811ea3f2b3 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:21.891879 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac9921-65ec-481c-8a0e-54811ea3f2b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:21.957572 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-a1ac9921-65ec-481c-8a0e-54811ea3f2b3 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:40:21.958760 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-a1ac9921-65ec-481c-8a0e-54811ea3f2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:22.056522 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 658/1336] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:40:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:40:22.357755 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462b89b7-40d2-4067-aa7d-aef669401f12 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:40:22.359330 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-462b89b7-40d2-4067-aa7d-aef669401f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:22.401452 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:40:22.477539 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462b89b7-40d2-4067-aa7d-aef669401f12 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:40:22.555547 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee6f65-a032-4082-9708-e226ac41420c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:40:22.556157 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee6f65-a032-4082-9708-e226ac41420c service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:22.556440 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee6f65-a032-4082-9708-e226ac41420c service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:22.556711 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee6f65-a032-4082-9708-e226ac41420c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:22.558334 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-13ee6f65-a032-4082-9708-e226ac41420c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:22.644529 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:40:22.670151 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ee6f65-a032-4082-9708-e226ac41420c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:40:22.671589 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ee6f65-a032-4082-9708-e226ac41420c service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:40:22.690635 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 679/1337] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:40:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:40:25.378486 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e1719-147a-4194-8ad4-025f82c895b1 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:25.379053 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e1719-147a-4194-8ad4-025f82c895b1 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:25.379053 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e1719-147a-4194-8ad4-025f82c895b1 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:25.380367 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 659/1338] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:40:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:40:25.386361 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e0438b-f02c-492b-a24d-61eb5e91744c None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:25.386812 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e0438b-f02c-492b-a24d-61eb5e91744c None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:25.386902 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e0438b-f02c-492b-a24d-61eb5e91744c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:25.453081 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-24e0438b-f02c-492b-a24d-61eb5e91744c None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:40:25.454276 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-24e0438b-f02c-492b-a24d-61eb5e91744c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:25.560208 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 680/1339] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:40:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:40:25.853278 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e1719-147a-4194-8ad4-025f82c895b1 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:40:25.854898 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-363e1719-147a-4194-8ad4-025f82c895b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:25.940494 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:40:25.969588 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e1719-147a-4194-8ad4-025f82c895b1 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:40:26.051476 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aa4112-38ad-4490-a45e-33c79ab4f3e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:40:26.052333 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa4112-38ad-4490-a45e-33c79ab4f3e0 service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:26.052333 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa4112-38ad-4490-a45e-33c79ab4f3e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:26.052616 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa4112-38ad-4490-a45e-33c79ab4f3e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:26.054275 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-a6aa4112-38ad-4490-a45e-33c79ab4f3e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:26.182878 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aa4112-38ad-4490-a45e-33c79ab4f3e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:40:26.184244 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aa4112-38ad-4490-a45e-33c79ab4f3e0 service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:40:26.208023 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 660/1340] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:40:25 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:40:30.602152 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174a2cd-68ed-4665-835f-e519f0c90656 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:30.602719 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174a2cd-68ed-4665-835f-e519f0c90656 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:30.602719 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174a2cd-68ed-4665-835f-e519f0c90656 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:30.605049 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 681/1341] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:40:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:40:30.611680 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428e7a2-f296-4704-bb07-b2950dd28f74 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:30.611680 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428e7a2-f296-4704-bb07-b2950dd28f74 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:30.611680 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428e7a2-f296-4704-bb07-b2950dd28f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:30.616584 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:40:30.682940 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-b428e7a2-f296-4704-bb07-b2950dd28f74 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:40:30.684167 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-b428e7a2-f296-4704-bb07-b2950dd28f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:30.785816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 661/1342] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:40:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:40:31.076135 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2174a2cd-68ed-4665-835f-e519f0c90656 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:40:31.077714 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-2174a2cd-68ed-4665-835f-e519f0c90656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:31.138394 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:40:31.199572 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2174a2cd-68ed-4665-835f-e519f0c90656 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:40:31.280052 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ef91a-8d4b-4d0a-995d-9e7ef404e96e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:40:31.280663 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ef91a-8d4b-4d0a-995d-9e7ef404e96e service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:31.280976 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ef91a-8d4b-4d0a-995d-9e7ef404e96e service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:31.281248 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ef91a-8d4b-4d0a-995d-9e7ef404e96e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:31.282849 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-931ef91a-8d4b-4d0a-995d-9e7ef404e96e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:31.390150 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:40:31.408609 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931ef91a-8d4b-4d0a-995d-9e7ef404e96e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:40:31.410126 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931ef91a-8d4b-4d0a-995d-9e7ef404e96e service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:40:31.429624 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 682/1343] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:40:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:40:34.733511 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1133f-b87c-46b3-b5ac-7854be759c5e None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:34.734256 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1133f-b87c-46b3-b5ac-7854be759c5e None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:34.734256 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1133f-b87c-46b3-b5ac-7854be759c5e None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:34.735519 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 662/1344] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:40:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:40:34.741381 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc61630-6dc6-439b-94e2-808046a047e1 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:34.742088 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc61630-6dc6-439b-94e2-808046a047e1 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:34.742216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc61630-6dc6-439b-94e2-808046a047e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:34.816658 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-4bc61630-6dc6-439b-94e2-808046a047e1 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:40:34.817949 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-4bc61630-6dc6-439b-94e2-808046a047e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:34.925964 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 683/1345] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:40:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:40:35.233699 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b1133f-b87c-46b3-b5ac-7854be759c5e None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:40:35.235237 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-89b1133f-b87c-46b3-b5ac-7854be759c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:35.338928 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:40:35.355914 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b1133f-b87c-46b3-b5ac-7854be759c5e None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:40:35.437341 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a929b0-e0a8-4dba-a3e5-0fe6710eaf1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:40:35.437866 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a929b0-e0a8-4dba-a3e5-0fe6710eaf1f service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:35.438140 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a929b0-e0a8-4dba-a3e5-0fe6710eaf1f service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:35.438403 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a929b0-e0a8-4dba-a3e5-0fe6710eaf1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:35.440302 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-f8a929b0-e0a8-4dba-a3e5-0fe6710eaf1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:35.574291 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a929b0-e0a8-4dba-a3e5-0fe6710eaf1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:40:35.576155 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a929b0-e0a8-4dba-a3e5-0fe6710eaf1f service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:40:35.596791 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 663/1346] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:40:35 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:40:40.183710 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741f585-67f6-44a3-b7ca-ca69c5aa8616 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:40.184238 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741f585-67f6-44a3-b7ca-ca69c5aa8616 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:40.184390 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741f585-67f6-44a3-b7ca-ca69c5aa8616 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:40.185873 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 684/1347] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:40:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:40:40.191968 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbe789b-781c-4e7e-b6a2-13e4c0962aeb None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:40.192488 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbe789b-781c-4e7e-b6a2-13e4c0962aeb None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:40.192644 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbe789b-781c-4e7e-b6a2-13e4c0962aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:40.210425 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:40:40.268101 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-afbe789b-781c-4e7e-b6a2-13e4c0962aeb None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:40:40.269351 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-afbe789b-781c-4e7e-b6a2-13e4c0962aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:40.383065 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 664/1348] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:40:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:40:40.672829 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a741f585-67f6-44a3-b7ca-ca69c5aa8616 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:40:40.675183 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-a741f585-67f6-44a3-b7ca-ca69c5aa8616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:40.748102 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:40:40.798071 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a741f585-67f6-44a3-b7ca-ca69c5aa8616 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:40:40.874671 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8025cb64-454c-4756-831d-96ca3595a001 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:40:40.875275 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8025cb64-454c-4756-831d-96ca3595a001 service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:40.875552 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8025cb64-454c-4756-831d-96ca3595a001 service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:40.875824 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8025cb64-454c-4756-831d-96ca3595a001 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:40.877492 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-8025cb64-454c-4756-831d-96ca3595a001 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:40.993180 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:40:40.998165 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8025cb64-454c-4756-831d-96ca3595a001 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:40:40.999614 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8025cb64-454c-4756-831d-96ca3595a001 service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:40:41.019364 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 685/1349] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:40:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:40:46.163376 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c850ce9-3df0-426f-b861-70a2274c1e44 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:46.163997 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c850ce9-3df0-426f-b861-70a2274c1e44 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:46.163997 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c850ce9-3df0-426f-b861-70a2274c1e44 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:46.172756 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 665/1350] 10.222.0.52 () {54 vars in 921 bytes} [Mon Aug 29 22:40:46 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:40:46.176869 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442aa60-55ac-4353-8c9a-d573e6ad04b1 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:46.177331 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442aa60-55ac-4353-8c9a-d573e6ad04b1 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:46.177576 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442aa60-55ac-4353-8c9a-d573e6ad04b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:46.272839 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-1442aa60-55ac-4353-8c9a-d573e6ad04b1 None None] MFA Rules not processed for user `d8fd0f6e5b144d2ca78b8d96722502a7`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:40:46.274077 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-1442aa60-55ac-4353-8c9a-d573e6ad04b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:46.470470 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 686/1351] 10.222.0.52 () {58 vars in 1018 bytes} [Mon Aug 29 22:40:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:40:50.381356 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=59 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:40:50.760051 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e25827-a697-414a-891c-8a85bfb332a2 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:50.761007 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e25827-a697-414a-891c-8a85bfb332a2 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:50.761328 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e25827-a697-414a-891c-8a85bfb332a2 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:50.762911 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 666/1352] 10.222.0.52 () {54 vars in 921 bytes} [Mon Aug 29 22:40:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:40:50.767858 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9fcca7-da41-45e9-9cf3-0e5ea6abc7fb None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:50.768339 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9fcca7-da41-45e9-9cf3-0e5ea6abc7fb None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:50.768619 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9fcca7-da41-45e9-9cf3-0e5ea6abc7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:50.831700 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:40:50.855832 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-8d9fcca7-da41-45e9-9cf3-0e5ea6abc7fb None None] MFA Rules not processed for user `d8fd0f6e5b144d2ca78b8d96722502a7`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:40:50.857200 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-8d9fcca7-da41-45e9-9cf3-0e5ea6abc7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:50.977931 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 687/1353] 10.222.0.52 () {58 vars in 1018 bytes} [Mon Aug 29 22:40:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:40:52.409193 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662ca7d2-c139-47d5-a78e-21c3de48bd91 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:52.409706 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662ca7d2-c139-47d5-a78e-21c3de48bd91 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:52.409893 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662ca7d2-c139-47d5-a78e-21c3de48bd91 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:52.411385 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 667/1354] 10.222.0.52 () {54 vars in 921 bytes} [Mon Aug 29 22:40:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:40:52.416560 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45457acb-3297-4a99-a119-7ad3c0b05523 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:52.417043 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45457acb-3297-4a99-a119-7ad3c0b05523 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:52.417315 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45457acb-3297-4a99-a119-7ad3c0b05523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:52.496811 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-45457acb-3297-4a99-a119-7ad3c0b05523 None None] MFA Rules not processed for user `d8fd0f6e5b144d2ca78b8d96722502a7`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:40:52.498024 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-45457acb-3297-4a99-a119-7ad3c0b05523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:52.542567 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:40:52.610669 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 688/1355] 10.222.0.52 () {58 vars in 1018 bytes} [Mon Aug 29 22:40:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:40:56.051129 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc7ae5-6b46-40b1-9a48-959f3aae60c6 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:56.052355 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc7ae5-6b46-40b1-9a48-959f3aae60c6 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:56.053003 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc7ae5-6b46-40b1-9a48-959f3aae60c6 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:56.054994 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 668/1356] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:40:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:40:56.061283 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8e395c-1fae-43c9-8559-f7e6f4945821 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:56.061743 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8e395c-1fae-43c9-8559-f7e6f4945821 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:56.061965 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8e395c-1fae-43c9-8559-f7e6f4945821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:56.140074 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-7e8e395c-1fae-43c9-8559-f7e6f4945821 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:40:56.142932 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-7e8e395c-1fae-43c9-8559-f7e6f4945821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:56.285595 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 689/1357] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:40:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:40:56.605287 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f673ed-2958-447c-be02-186fe69063c2 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:56.606413 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f673ed-2958-447c-be02-186fe69063c2 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:56.607716 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f673ed-2958-447c-be02-186fe69063c2 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:56.612238 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 669/1358] 10.222.0.52 () {54 vars in 962 bytes} [Mon Aug 29 22:40:56 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:40:56.621331 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5350348-7b94-43b9-a343-e75df467be45 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:56.621331 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5350348-7b94-43b9-a343-e75df467be45 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:56.621331 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5350348-7b94-43b9-a343-e75df467be45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:56.638272 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:40:56.709061 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-d5350348-7b94-43b9-a343-e75df467be45 None None] MFA Rules not processed for user `bdbf961e02934e88af3a72d6218f32fa`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:40:56.710398 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-d5350348-7b94-43b9-a343-e75df467be45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:56.837260 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 690/1359] 10.222.0.52 () {58 vars in 1059 bytes} [Mon Aug 29 22:40:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:40:56.845937 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f673ed-2958-447c-be02-186fe69063c2 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:40:56.847463 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-04f673ed-2958-447c-be02-186fe69063c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:57.016814 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f673ed-2958-447c-be02-186fe69063c2 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:40:57.099359 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:40:57.141514 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6e283e-93ef-4f65-adb7-1ea9930a567e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:40:57.143158 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e283e-93ef-4f65-adb7-1ea9930a567e service nova] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:57.143158 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e283e-93ef-4f65-adb7-1ea9930a567e service nova] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:57.143158 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e283e-93ef-4f65-adb7-1ea9930a567e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:57.144264 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-0b6e283e-93ef-4f65-adb7-1ea9930a567e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:57.266082 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6e283e-93ef-4f65-adb7-1ea9930a567e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:40:57.267969 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6e283e-93ef-4f65-adb7-1ea9930a567e service nova] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:40:57.306157 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 670/1360] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:40:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:40:59.525196 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b72bf9-1c20-4a22-b0a3-7fdd88e51662 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:59.525940 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b72bf9-1c20-4a22-b0a3-7fdd88e51662 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:59.525940 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b72bf9-1c20-4a22-b0a3-7fdd88e51662 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:59.527400 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 691/1361] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:40:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:40:59.535949 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d52a5-f325-4737-9ee4-6b0eb8ff6cd6 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:59.535949 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d52a5-f325-4737-9ee4-6b0eb8ff6cd6 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:59.535949 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d52a5-f325-4737-9ee4-6b0eb8ff6cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:59.610645 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-dd1d52a5-f325-4737-9ee4-6b0eb8ff6cd6 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:40:59.611984 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-dd1d52a5-f325-4737-9ee4-6b0eb8ff6cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:59.665348 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:40:59.736808 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 671/1362] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:40:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:41:00.026744 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b72bf9-1c20-4a22-b0a3-7fdd88e51662 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:41:00.028481 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-b9b72bf9-1c20-4a22-b0a3-7fdd88e51662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:00.101493 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:00.203687 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b72bf9-1c20-4a22-b0a3-7fdd88e51662 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:41:00.331512 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4817da-e0d9-43a8-89c5-f9b28c654b2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:41:00.332463 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4817da-e0d9-43a8-89c5-f9b28c654b2c service nova] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:00.332984 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4817da-e0d9-43a8-89c5-f9b28c654b2c service nova] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:00.333402 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4817da-e0d9-43a8-89c5-f9b28c654b2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:00.335206 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-ee4817da-e0d9-43a8-89c5-f9b28c654b2c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:00.361447 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:00.442114 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4817da-e0d9-43a8-89c5-f9b28c654b2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:41:00.443748 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4817da-e0d9-43a8-89c5-f9b28c654b2c service nova] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:41:00.468102 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 692/1363] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:41:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:41:02.893667 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86534a58-d16b-439a-8e8f-dcbf74b04fee None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:02.894529 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86534a58-d16b-439a-8e8f-dcbf74b04fee None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:02.894967 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86534a58-d16b-439a-8e8f-dcbf74b04fee None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:02.896932 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 672/1364] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:41:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:41:02.902687 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ca415-1474-4f6a-9baa-14c52ada14ac None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:02.903146 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ca415-1474-4f6a-9baa-14c52ada14ac None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:02.903229 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ca415-1474-4f6a-9baa-14c52ada14ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:02.989370 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-3f7ca415-1474-4f6a-9baa-14c52ada14ac None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:41:02.990595 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-3f7ca415-1474-4f6a-9baa-14c52ada14ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:03.096394 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:03.099394 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 693/1365] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:41:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:41:03.395212 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc2aa8-ee69-447a-bdfc-9f521c1439fe None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:03.395503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc2aa8-ee69-447a-bdfc-9f521c1439fe None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:03.395787 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc2aa8-ee69-447a-bdfc-9f521c1439fe None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:03.397422 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 673/1366] 10.222.0.52 () {54 vars in 962 bytes} [Mon Aug 29 22:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:41:03.404941 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba394c-89ee-4997-9af6-3d73b7cf4940 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:03.405236 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba394c-89ee-4997-9af6-3d73b7cf4940 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:03.408920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba394c-89ee-4997-9af6-3d73b7cf4940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:03.487958 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-28ba394c-89ee-4997-9af6-3d73b7cf4940 None None] MFA Rules not processed for user `bdbf961e02934e88af3a72d6218f32fa`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:41:03.489313 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-28ba394c-89ee-4997-9af6-3d73b7cf4940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:03.628771 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 694/1367] 10.222.0.52 () {58 vars in 1059 bytes} [Mon Aug 29 22:41:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:41:03.641622 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bc2aa8-ee69-447a-bdfc-9f521c1439fe None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:41:03.643053 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-52bc2aa8-ee69-447a-bdfc-9f521c1439fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:03.814208 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bc2aa8-ee69-447a-bdfc-9f521c1439fe None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:41:03.836333 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:03.940622 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40cbe49-e13e-4c83-9a8d-4c91db079d43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:41:03.942098 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40cbe49-e13e-4c83-9a8d-4c91db079d43 service nova] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:03.942607 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40cbe49-e13e-4c83-9a8d-4c91db079d43 service nova] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:03.943077 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40cbe49-e13e-4c83-9a8d-4c91db079d43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:03.945226 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-c40cbe49-e13e-4c83-9a8d-4c91db079d43 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:04.047299 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40cbe49-e13e-4c83-9a8d-4c91db079d43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:41:04.048937 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40cbe49-e13e-4c83-9a8d-4c91db079d43 service nova] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:41:04.068755 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 674/1368] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:41:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:41:04.599928 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eba648-2e3f-449a-9796-b62c93059f56 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:04.600540 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eba648-2e3f-449a-9796-b62c93059f56 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:04.600540 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eba648-2e3f-449a-9796-b62c93059f56 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:04.602620 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 695/1369] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:41:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:41:04.608274 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dda9ec-0212-46b6-8223-6c1e2a45d87a None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:04.613664 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dda9ec-0212-46b6-8223-6c1e2a45d87a None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:04.613664 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dda9ec-0212-46b6-8223-6c1e2a45d87a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:04.644094 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:04.702284 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-68dda9ec-0212-46b6-8223-6c1e2a45d87a None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:41:04.703516 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-68dda9ec-0212-46b6-8223-6c1e2a45d87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:04.813460 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 675/1370] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:41:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:41:05.418000 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda4728-9742-45b1-b010-09f800a771e7 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:05.418000 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda4728-9742-45b1-b010-09f800a771e7 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:05.418000 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda4728-9742-45b1-b010-09f800a771e7 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:05.420642 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 696/1371] 10.222.0.52 () {54 vars in 964 bytes} [Mon Aug 29 22:41:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:41:05.425515 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86af30c-dc60-452d-91c1-9e4dc828bc0c None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:05.426596 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86af30c-dc60-452d-91c1-9e4dc828bc0c None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:05.426596 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86af30c-dc60-452d-91c1-9e4dc828bc0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:05.512562 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-b86af30c-dc60-452d-91c1-9e4dc828bc0c None None] MFA Rules not processed for user `6e782c8acaaa42c4bd78ddc7c8cc654b`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:41:05.513977 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-b86af30c-dc60-452d-91c1-9e4dc828bc0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:05.563232 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:05.595978 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 676/1372] 10.222.0.52 () {58 vars in 1061 bytes} [Mon Aug 29 22:41:05 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:41:05.606967 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda4728-9742-45b1-b010-09f800a771e7 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:41:05.608618 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-ebda4728-9742-45b1-b010-09f800a771e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:05.673180 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:05.739251 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda4728-9742-45b1-b010-09f800a771e7 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:41:05.829848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5176a201-6e02-4972-b7b9-b07b812ddedb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:41:05.830567 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176a201-6e02-4972-b7b9-b07b812ddedb service cinder] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:05.830946 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176a201-6e02-4972-b7b9-b07b812ddedb service cinder] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:05.830946 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176a201-6e02-4972-b7b9-b07b812ddedb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:05.832624 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-5176a201-6e02-4972-b7b9-b07b812ddedb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:05.948870 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:05.967540 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5176a201-6e02-4972-b7b9-b07b812ddedb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:41:05.967540 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5176a201-6e02-4972-b7b9-b07b812ddedb service cinder] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:41:05.993085 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 697/1373] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:41:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:41:06.229786 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fecf205-f28f-4404-866f-422e80d6175c None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:06.230108 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fecf205-f28f-4404-866f-422e80d6175c None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:06.230334 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fecf205-f28f-4404-866f-422e80d6175c None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:06.231728 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 677/1374] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:41:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:41:06.237922 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12402d11-72c0-40f0-ba0c-2c60d6d9e6a0 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:06.238330 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12402d11-72c0-40f0-ba0c-2c60d6d9e6a0 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:06.238691 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12402d11-72c0-40f0-ba0c-2c60d6d9e6a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:06.308555 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-12402d11-72c0-40f0-ba0c-2c60d6d9e6a0 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:41:06.309798 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-12402d11-72c0-40f0-ba0c-2c60d6d9e6a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:06.410104 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 698/1375] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:41:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:41:06.703217 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fecf205-f28f-4404-866f-422e80d6175c None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:41:06.704817 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-3fecf205-f28f-4404-866f-422e80d6175c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:06.861357 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fecf205-f28f-4404-866f-422e80d6175c None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:41:06.915599 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:06.982082 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09660079-d39d-4904-99fd-6f60b01886d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:41:06.982366 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09660079-d39d-4904-99fd-6f60b01886d3 service nova] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:06.982464 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09660079-d39d-4904-99fd-6f60b01886d3 service nova] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:06.982691 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09660079-d39d-4904-99fd-6f60b01886d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:06.984312 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-09660079-d39d-4904-99fd-6f60b01886d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:07.081146 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09660079-d39d-4904-99fd-6f60b01886d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:41:07.082667 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09660079-d39d-4904-99fd-6f60b01886d3 service nova] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:41:07.101475 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 678/1376] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:41:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:41:09.193302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7acf2e-adc6-4e4c-ba21-3c2516bfe9ae None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:09.193755 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7acf2e-adc6-4e4c-ba21-3c2516bfe9ae None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:09.193856 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7acf2e-adc6-4e4c-ba21-3c2516bfe9ae None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:09.195504 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 699/1377] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:41:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:41:09.201742 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56160163-41d3-4db9-9242-be03307ee90f None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:09.202238 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56160163-41d3-4db9-9242-be03307ee90f None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:09.202343 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56160163-41d3-4db9-9242-be03307ee90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:09.271898 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-56160163-41d3-4db9-9242-be03307ee90f None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:41:09.273125 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-56160163-41d3-4db9-9242-be03307ee90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:09.280065 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:09.377882 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 679/1378] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:41:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:41:09.684513 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7acf2e-adc6-4e4c-ba21-3c2516bfe9ae None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:41:09.686223 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-5a7acf2e-adc6-4e4c-ba21-3c2516bfe9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:09.764190 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:09.878948 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7acf2e-adc6-4e4c-ba21-3c2516bfe9ae None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:41:10.015693 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e7756-b9b3-42c3-9e02-13feffa35ce4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:41:10.016300 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e7756-b9b3-42c3-9e02-13feffa35ce4 service nova] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:10.016577 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e7756-b9b3-42c3-9e02-13feffa35ce4 service nova] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:10.016882 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e7756-b9b3-42c3-9e02-13feffa35ce4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:10.018498 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-632e7756-b9b3-42c3-9e02-13feffa35ce4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:10.038700 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:10.121426 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632e7756-b9b3-42c3-9e02-13feffa35ce4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:41:10.122952 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632e7756-b9b3-42c3-9e02-13feffa35ce4 service nova] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:41:10.143074 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 700/1379] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:41:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:41:12.324969 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c09622-25b2-4f01-a563-a96371570a7a None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:12.325537 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c09622-25b2-4f01-a563-a96371570a7a None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:12.325537 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c09622-25b2-4f01-a563-a96371570a7a None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:12.326829 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 680/1380] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:41:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:41:12.332810 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598e865-d618-4c92-9241-5495dec35bfd None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:12.333291 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598e865-d618-4c92-9241-5495dec35bfd None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:12.333551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598e865-d618-4c92-9241-5495dec35bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:12.406170 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-a598e865-d618-4c92-9241-5495dec35bfd None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:41:12.407393 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-a598e865-d618-4c92-9241-5495dec35bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:12.505692 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:12.514016 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 701/1381] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:41:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:41:12.808036 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c09622-25b2-4f01-a563-a96371570a7a None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:41:12.809590 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-78c09622-25b2-4f01-a563-a96371570a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:12.963947 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:12.975532 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c09622-25b2-4f01-a563-a96371570a7a None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:41:13.108029 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902a7093-412f-47c7-bca7-0abd8f42c21e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:41:13.108545 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902a7093-412f-47c7-bca7-0abd8f42c21e service nova] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:13.108855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902a7093-412f-47c7-bca7-0abd8f42c21e service nova] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:13.109128 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902a7093-412f-47c7-bca7-0abd8f42c21e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:13.110722 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-902a7093-412f-47c7-bca7-0abd8f42c21e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:13.204773 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902a7093-412f-47c7-bca7-0abd8f42c21e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:41:13.206382 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902a7093-412f-47c7-bca7-0abd8f42c21e service nova] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:41:13.223216 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:13.233637 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 681/1382] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:41:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:41:15.877084 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2189a3-e059-469b-8577-4da7aa11d348 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:15.877591 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2189a3-e059-469b-8577-4da7aa11d348 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:15.877591 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2189a3-e059-469b-8577-4da7aa11d348 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:15.878986 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 702/1383] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:41:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:41:15.884554 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d02190-80ae-4034-9832-ac82fa1777cb None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:15.885153 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d02190-80ae-4034-9832-ac82fa1777cb None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:15.885153 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d02190-80ae-4034-9832-ac82fa1777cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:15.960586 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-49d02190-80ae-4034-9832-ac82fa1777cb None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:41:15.961892 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-49d02190-80ae-4034-9832-ac82fa1777cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:16.080438 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 682/1384] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:41:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:41:16.373105 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2189a3-e059-469b-8577-4da7aa11d348 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:41:16.374727 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-6e2189a3-e059-469b-8577-4da7aa11d348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:16.564137 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2189a3-e059-469b-8577-4da7aa11d348 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:41:16.647383 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:16.708651 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cdab4a-5392-4537-998d-649ee9995208 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:41:16.709650 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cdab4a-5392-4537-998d-649ee9995208 service nova] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:16.709975 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cdab4a-5392-4537-998d-649ee9995208 service nova] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:16.710338 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cdab4a-5392-4537-998d-649ee9995208 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:16.712431 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-46cdab4a-5392-4537-998d-649ee9995208 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:16.824163 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cdab4a-5392-4537-998d-649ee9995208 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:41:16.826240 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cdab4a-5392-4537-998d-649ee9995208 service nova] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:41:16.846724 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 703/1385] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:41:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:41:18.796116 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2cdaa-5b65-4a35-9b6a-bc0c400cddf1 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:18.796837 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2cdaa-5b65-4a35-9b6a-bc0c400cddf1 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:18.796837 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2cdaa-5b65-4a35-9b6a-bc0c400cddf1 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:18.797911 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 683/1386] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:41:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:41:18.803382 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191e5d7-19f0-4926-84ff-ac378aad5709 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:18.803882 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191e5d7-19f0-4926-84ff-ac378aad5709 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:18.804015 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191e5d7-19f0-4926-84ff-ac378aad5709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:18.884147 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-4191e5d7-19f0-4926-84ff-ac378aad5709 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:41:18.886703 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-4191e5d7-19f0-4926-84ff-ac378aad5709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:18.928172 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:19.007329 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 704/1387] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:41:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:41:19.379744 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f2cdaa-5b65-4a35-9b6a-bc0c400cddf1 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:41:19.381737 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-b5f2cdaa-5b65-4a35-9b6a-bc0c400cddf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:19.477091 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:19.566590 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f2cdaa-5b65-4a35-9b6a-bc0c400cddf1 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:41:19.732190 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d693eaca-065e-48a8-bb1b-19a4b6883a21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:41:19.732962 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693eaca-065e-48a8-bb1b-19a4b6883a21 service nova] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:19.733344 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693eaca-065e-48a8-bb1b-19a4b6883a21 service nova] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:19.733688 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693eaca-065e-48a8-bb1b-19a4b6883a21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:19.735741 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-d693eaca-065e-48a8-bb1b-19a4b6883a21 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:19.787148 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:19.845603 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d693eaca-065e-48a8-bb1b-19a4b6883a21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:41:19.847212 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d693eaca-065e-48a8-bb1b-19a4b6883a21 service nova] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:41:19.866247 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 684/1388] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:41:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:41:21.799287 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62325375-05bc-4c14-a3ae-356e5ea3cb38 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:21.799823 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62325375-05bc-4c14-a3ae-356e5ea3cb38 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:21.799823 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62325375-05bc-4c14-a3ae-356e5ea3cb38 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:21.801111 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 705/1389] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:41:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:41:21.806444 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed12e1-9997-45fa-874a-9bd122772285 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:21.806919 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed12e1-9997-45fa-874a-9bd122772285 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:21.807026 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed12e1-9997-45fa-874a-9bd122772285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:21.875457 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-21ed12e1-9997-45fa-874a-9bd122772285 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:41:21.876761 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-21ed12e1-9997-45fa-874a-9bd122772285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:21.977000 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 685/1390] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:41:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:41:21.989446 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f041bf-8960-497f-be70-c309cac26364 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:21.989731 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f041bf-8960-497f-be70-c309cac26364 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:21.990011 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f041bf-8960-497f-be70-c309cac26364 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:21.991351 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 706/1391] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:41:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:41:21.996618 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e1438e-7cce-4ee1-abff-e764acda8507 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:21.996986 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e1438e-7cce-4ee1-abff-e764acda8507 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:21.997294 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e1438e-7cce-4ee1-abff-e764acda8507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:22.013404 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:22.065926 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-c2e1438e-7cce-4ee1-abff-e764acda8507 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:41:22.067048 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-c2e1438e-7cce-4ee1-abff-e764acda8507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:22.170746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 686/1392] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:41:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:41:22.278789 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c1e8b-0024-486e-af31-3d837cb762cf None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:22.284119 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c1e8b-0024-486e-af31-3d837cb762cf None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:22.284119 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c1e8b-0024-486e-af31-3d837cb762cf None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:22.284119 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 707/1393] 10.222.0.52 () {54 vars in 967 bytes} [Mon Aug 29 22:41:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:41:22.286155 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6321ab-9683-4886-b6b3-6d2ddf071286 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:22.286511 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6321ab-9683-4886-b6b3-6d2ddf071286 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:22.286882 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6321ab-9683-4886-b6b3-6d2ddf071286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:22.368997 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-6c6321ab-9683-4886-b6b3-6d2ddf071286 None None] MFA Rules not processed for user `9c2569e7cd774199aa5750fd6a850060`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:41:22.371009 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-6c6321ab-9683-4886-b6b3-6d2ddf071286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:22.406883 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:22.449228 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 687/1394] 10.222.0.52 () {58 vars in 1064 bytes} [Mon Aug 29 22:41:22 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:41:22.457829 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54c1e8b-0024-486e-af31-3d837cb762cf None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:41:22.459505 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-c54c1e8b-0024-486e-af31-3d837cb762cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:22.463977 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6321ab-9683-4886-b6b3-6d2ddf071286 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:41:22.465288 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-6c6321ab-9683-4886-b6b3-6d2ddf071286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:22.580460 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54c1e8b-0024-486e-af31-3d837cb762cf None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:41:22.603102 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6321ab-9683-4886-b6b3-6d2ddf071286 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:41:22.652006 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:22.670062 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2270065-c34f-48ec-82ed-a88dcb83e562 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c2569e7cd774199aa5750fd6a850060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:41:22.670675 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2270065-c34f-48ec-82ed-a88dcb83e562 service glance] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:22.671107 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2270065-c34f-48ec-82ed-a88dcb83e562 service glance] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:22.671380 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2270065-c34f-48ec-82ed-a88dcb83e562 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:22.673046 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-f2270065-c34f-48ec-82ed-a88dcb83e562 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:22.674496 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:22.730122 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cceb04-1222-4850-99ba-e0af2389901c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:41:22.730872 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cceb04-1222-4850-99ba-e0af2389901c service nova] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:22.731158 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cceb04-1222-4850-99ba-e0af2389901c service nova] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:22.731433 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cceb04-1222-4850-99ba-e0af2389901c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:22.733098 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-e4cceb04-1222-4850-99ba-e0af2389901c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:22.792248 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2270065-c34f-48ec-82ed-a88dcb83e562 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:41:22.793720 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2270065-c34f-48ec-82ed-a88dcb83e562 service glance] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:41:22.812351 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 708/1395] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:41:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:41:22.826910 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cceb04-1222-4850-99ba-e0af2389901c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:41:22.828490 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cceb04-1222-4850-99ba-e0af2389901c service nova] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:41:22.846720 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 688/1396] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:41:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:41:23.750367 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e1cc4d-4820-4a21-a39b-faffb3a7ee16 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:23.750666 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e1cc4d-4820-4a21-a39b-faffb3a7ee16 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:23.750950 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e1cc4d-4820-4a21-a39b-faffb3a7ee16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:23.821970 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-a2e1cc4d-4820-4a21-a39b-faffb3a7ee16 None None] MFA Rules not processed for user `f77dfe78b1a8405084c06d9acee96511`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:41:23.823101 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-a2e1cc4d-4820-4a21-a39b-faffb3a7ee16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:23.865771 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:23.912729 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 709/1397] 10.222.0.52 () {58 vars in 1009 bytes} [Mon Aug 29 22:41:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:41:23.961367 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce177e-8033-4463-b54b-031c4442ab6a None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:23.961830 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce177e-8033-4463-b54b-031c4442ab6a None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:23.961958 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce177e-8033-4463-b54b-031c4442ab6a None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:23.963754 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 689/1398] 10.222.0.52 () {54 vars in 943 bytes} [Mon Aug 29 22:41:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:41:23.970515 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a453cd2-216f-4927-b168-c76ad12d5f9a None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:23.970808 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a453cd2-216f-4927-b168-c76ad12d5f9a None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:23.971092 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a453cd2-216f-4927-b168-c76ad12d5f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:24.051188 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-7a453cd2-216f-4927-b168-c76ad12d5f9a None None] MFA Rules not processed for user `57c175ac90e545358746176972ff39a3`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:41:24.052399 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-7a453cd2-216f-4927-b168-c76ad12d5f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:24.164185 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 710/1399] 10.222.0.52 () {58 vars in 1040 bytes} [Mon Aug 29 22:41:23 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:41:24.174025 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce177e-8033-4463-b54b-031c4442ab6a None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:41:24.175591 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-71ce177e-8033-4463-b54b-031c4442ab6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:24.255758 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:24.317563 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce177e-8033-4463-b54b-031c4442ab6a None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:41:24.431535 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560da4f8-0522-48ec-8e0b-84a16f54c7de service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57c175ac90e545358746176972ff39a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:41:24.432086 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560da4f8-0522-48ec-8e0b-84a16f54c7de service swift] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:24.432366 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560da4f8-0522-48ec-8e0b-84a16f54c7de service swift] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:24.432769 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560da4f8-0522-48ec-8e0b-84a16f54c7de service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:24.434421 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-560da4f8-0522-48ec-8e0b-84a16f54c7de service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:24.498104 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:24.521454 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560da4f8-0522-48ec-8e0b-84a16f54c7de service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:41:24.523049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560da4f8-0522-48ec-8e0b-84a16f54c7de service swift] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:41:24.532329 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 690/1400] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Aug 29 22:41:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 22:41:24.940775 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe8ff55-4e3c-4466-9e4a-ad2feae48e7e None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:24.941250 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe8ff55-4e3c-4466-9e4a-ad2feae48e7e None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:24.941402 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe8ff55-4e3c-4466-9e4a-ad2feae48e7e None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:24.942718 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 711/1401] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:41:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:41:24.948533 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876eb90b-9216-453d-aa83-e364526760b1 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:24.948853 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876eb90b-9216-453d-aa83-e364526760b1 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:24.949138 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876eb90b-9216-453d-aa83-e364526760b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:25.019936 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-876eb90b-9216-453d-aa83-e364526760b1 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:41:25.021177 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-876eb90b-9216-453d-aa83-e364526760b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:25.136222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 691/1402] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:41:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:41:25.430921 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe8ff55-4e3c-4466-9e4a-ad2feae48e7e None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:41:25.432581 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-2fe8ff55-4e3c-4466-9e4a-ad2feae48e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:25.447792 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:25.614917 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe8ff55-4e3c-4466-9e4a-ad2feae48e7e None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:41:25.718111 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:25.749198 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6802b727-cc9a-4bc8-936e-9451ce10d124 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:41:25.749819 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802b727-cc9a-4bc8-936e-9451ce10d124 service nova] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:25.750104 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802b727-cc9a-4bc8-936e-9451ce10d124 service nova] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:25.750381 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802b727-cc9a-4bc8-936e-9451ce10d124 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:25.752034 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-6802b727-cc9a-4bc8-936e-9451ce10d124 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:25.855758 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6802b727-cc9a-4bc8-936e-9451ce10d124 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:41:25.857405 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6802b727-cc9a-4bc8-936e-9451ce10d124 service nova] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:41:25.876200 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 712/1403] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:41:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:41:28.024225 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78579dd2-d19b-4786-8441-351550e6a376 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:28.024876 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78579dd2-d19b-4786-8441-351550e6a376 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:28.024876 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78579dd2-d19b-4786-8441-351550e6a376 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:28.026876 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 692/1404] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:41:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:41:28.031955 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cfba22-71e9-470b-b775-1346df721f4a None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:28.032386 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cfba22-71e9-470b-b775-1346df721f4a None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:28.032512 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cfba22-71e9-470b-b775-1346df721f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:28.116586 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-e5cfba22-71e9-470b-b775-1346df721f4a None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:41:28.117854 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-e5cfba22-71e9-470b-b775-1346df721f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:28.182527 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:28.244065 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 713/1405] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:41:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:41:28.556332 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78579dd2-d19b-4786-8441-351550e6a376 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:41:28.557977 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-78579dd2-d19b-4786-8441-351550e6a376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:28.625959 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:28.751668 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78579dd2-d19b-4786-8441-351550e6a376 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:41:28.911763 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b78e51a-4c14-4667-8388-071ca78d9de6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:41:28.912278 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b78e51a-4c14-4667-8388-071ca78d9de6 service nova] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:28.912556 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b78e51a-4c14-4667-8388-071ca78d9de6 service nova] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:28.912857 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b78e51a-4c14-4667-8388-071ca78d9de6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:28.914534 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-3b78e51a-4c14-4667-8388-071ca78d9de6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:28.928004 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:29.015799 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b78e51a-4c14-4667-8388-071ca78d9de6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:41:29.017426 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b78e51a-4c14-4667-8388-071ca78d9de6 service nova] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:41:29.036298 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 693/1406] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:41:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:41:31.207928 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a1d847-fb5e-4bda-b4e1-88065c9498c9 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:31.208519 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a1d847-fb5e-4bda-b4e1-88065c9498c9 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:31.208519 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a1d847-fb5e-4bda-b4e1-88065c9498c9 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:31.209858 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 714/1407] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:41:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:41:31.215121 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f45d4a-0cb4-4907-99bd-f1ae54e4a806 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:31.215563 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f45d4a-0cb4-4907-99bd-f1ae54e4a806 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:31.215661 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f45d4a-0cb4-4907-99bd-f1ae54e4a806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:31.294617 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-b7f45d4a-0cb4-4907-99bd-f1ae54e4a806 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:41:31.295851 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-b7f45d4a-0cb4-4907-99bd-f1ae54e4a806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:31.395341 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:31.407986 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 694/1408] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:41:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:41:31.704237 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a1d847-fb5e-4bda-b4e1-88065c9498c9 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:41:31.707722 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-46a1d847-fb5e-4bda-b4e1-88065c9498c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:31.912364 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a1d847-fb5e-4bda-b4e1-88065c9498c9 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:41:31.977178 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:32.081721 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279f58dc-1fe2-4b5e-bed4-33c5767e2778 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:41:32.082321 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f58dc-1fe2-4b5e-bed4-33c5767e2778 service nova] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:32.082726 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f58dc-1fe2-4b5e-bed4-33c5767e2778 service nova] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:32.082899 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f58dc-1fe2-4b5e-bed4-33c5767e2778 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:32.084516 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-279f58dc-1fe2-4b5e-bed4-33c5767e2778 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:32.189300 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279f58dc-1fe2-4b5e-bed4-33c5767e2778 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:41:32.191829 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279f58dc-1fe2-4b5e-bed4-33c5767e2778 service nova] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:41:32.214128 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 715/1409] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:41:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:41:41.434604 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=4 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:42.209538 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=83 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:48.801892 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc11379f-4e40-4ddb-aab3-53f855f904aa None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:48.802408 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc11379f-4e40-4ddb-aab3-53f855f904aa None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:48.802408 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc11379f-4e40-4ddb-aab3-53f855f904aa None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:48.803709 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 695/1410] 10.222.0.52 () {54 vars in 918 bytes} [Mon Aug 29 22:41:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:41:48.808865 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e97da-59af-451b-9d9f-008a5fdbc8ad None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:48.809294 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e97da-59af-451b-9d9f-008a5fdbc8ad None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:48.809413 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e97da-59af-451b-9d9f-008a5fdbc8ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:48.886361 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-653e97da-59af-451b-9d9f-008a5fdbc8ad None None] MFA Rules not processed for user `d8fd0f6e5b144d2ca78b8d96722502a7`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:41:48.887583 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-653e97da-59af-451b-9d9f-008a5fdbc8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:48.994546 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 716/1411] 10.222.0.52 () {58 vars in 1015 bytes} [Mon Aug 29 22:41:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:41:49.009153 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749fb7a5-76d7-4be9-8775-d491d67287a0 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:49.009442 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749fb7a5-76d7-4be9-8775-d491d67287a0 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:49.009716 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749fb7a5-76d7-4be9-8775-d491d67287a0 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:49.011698 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 696/1412] 10.222.0.52 () {54 vars in 918 bytes} [Mon Aug 29 22:41:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:41:49.016382 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a223996e-3b97-4b62-b3d5-5c4cf80d4d66 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:49.016671 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a223996e-3b97-4b62-b3d5-5c4cf80d4d66 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:49.016974 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a223996e-3b97-4b62-b3d5-5c4cf80d4d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:49.090633 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:49.093686 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-a223996e-3b97-4b62-b3d5-5c4cf80d4d66 None None] MFA Rules not processed for user `bdbf961e02934e88af3a72d6218f32fa`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:41:49.094907 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-a223996e-3b97-4b62-b3d5-5c4cf80d4d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:49.212358 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 717/1413] 10.222.0.52 () {58 vars in 1015 bytes} [Mon Aug 29 22:41:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:41:51.664507 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f0523-3244-4869-a588-0ce5aaf20216 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:51.665093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f0523-3244-4869-a588-0ce5aaf20216 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:51.665093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f0523-3244-4869-a588-0ce5aaf20216 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:51.666955 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 697/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 22:41:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:41:51.672194 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4407bc60-2ea2-4b9e-ab09-4a5dc7fcf207 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:51.672613 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4407bc60-2ea2-4b9e-ab09-4a5dc7fcf207 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:51.672779 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4407bc60-2ea2-4b9e-ab09-4a5dc7fcf207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:51.742286 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-4407bc60-2ea2-4b9e-ab09-4a5dc7fcf207 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:41:51.743505 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-4407bc60-2ea2-4b9e-ab09-4a5dc7fcf207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:51.798688 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:51.843569 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 718/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 22:41:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:41:52.131127 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4f0523-3244-4869-a588-0ce5aaf20216 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:41:52.133574 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-ab4f0523-3244-4869-a588-0ce5aaf20216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:52.292550 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4f0523-3244-4869-a588-0ce5aaf20216 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:41:52.367840 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:52.409288 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07e7faa-d860-4252-a1c0-580ca5caf62f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:41:52.409896 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e7faa-d860-4252-a1c0-580ca5caf62f service nova] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:52.410180 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e7faa-d860-4252-a1c0-580ca5caf62f service nova] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:52.410452 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e7faa-d860-4252-a1c0-580ca5caf62f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:52.412146 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-c07e7faa-d860-4252-a1c0-580ca5caf62f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:52.501648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07e7faa-d860-4252-a1c0-580ca5caf62f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:41:52.503275 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07e7faa-d860-4252-a1c0-580ca5caf62f service nova] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:41:52.523308 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 698/1416] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:41:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:41:53.421736 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062d942-0899-4fa7-b7e2-58c1abe07b58 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:53.422252 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062d942-0899-4fa7-b7e2-58c1abe07b58 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:53.422252 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062d942-0899-4fa7-b7e2-58c1abe07b58 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:53.423532 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 719/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 22:41:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:41:53.428342 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f2f81f-c724-458e-a6ce-cddb2f539324 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:53.428810 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f2f81f-c724-458e-a6ce-cddb2f539324 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:53.428933 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f2f81f-c724-458e-a6ce-cddb2f539324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:53.512278 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-46f2f81f-c724-458e-a6ce-cddb2f539324 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:41:53.513527 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-46f2f81f-c724-458e-a6ce-cddb2f539324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:53.543072 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:53.613960 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 699/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 22:41:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:41:53.763958 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1062d942-0899-4fa7-b7e2-58c1abe07b58 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:41:53.765549 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-1062d942-0899-4fa7-b7e2-58c1abe07b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:53.913819 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1062d942-0899-4fa7-b7e2-58c1abe07b58 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:41:53.954842 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:54.029947 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c10a4-b670-4289-91d7-d52982abbf5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:41:54.030486 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c10a4-b670-4289-91d7-d52982abbf5c service nova] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:54.030764 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c10a4-b670-4289-91d7-d52982abbf5c service nova] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:54.031085 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c10a4-b670-4289-91d7-d52982abbf5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:54.032684 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-662c10a4-b670-4289-91d7-d52982abbf5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:54.123224 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c10a4-b670-4289-91d7-d52982abbf5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:41:54.124839 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c10a4-b670-4289-91d7-d52982abbf5c service nova] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:41:54.143693 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 720/1419] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:41:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:41:55.043220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eb3f96-d5fd-417f-a8b2-d92d6e89e149 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:55.043819 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eb3f96-d5fd-417f-a8b2-d92d6e89e149 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:55.043819 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eb3f96-d5fd-417f-a8b2-d92d6e89e149 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:55.045173 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 700/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 22:41:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:41:55.050103 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5cbf6-bc96-44c1-a63e-522be6bf1eea None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:55.050535 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5cbf6-bc96-44c1-a63e-522be6bf1eea None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:55.050721 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5cbf6-bc96-44c1-a63e-522be6bf1eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:55.106445 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:55.244649 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-87d5cbf6-bc96-44c1-a63e-522be6bf1eea None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:41:55.245990 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-87d5cbf6-bc96-44c1-a63e-522be6bf1eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:55.342536 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 721/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 22:41:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:41:55.496546 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eb3f96-d5fd-417f-a8b2-d92d6e89e149 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:41:55.496891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-25eb3f96-d5fd-417f-a8b2-d92d6e89e149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:55.650016 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eb3f96-d5fd-417f-a8b2-d92d6e89e149 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:41:55.662815 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:55.765107 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a8ca24-2fe9-4852-b245-436403b5081c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:41:55.765628 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8ca24-2fe9-4852-b245-436403b5081c service nova] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:55.765904 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8ca24-2fe9-4852-b245-436403b5081c service nova] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:55.766171 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8ca24-2fe9-4852-b245-436403b5081c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:55.767757 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-59a8ca24-2fe9-4852-b245-436403b5081c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:55.857433 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a8ca24-2fe9-4852-b245-436403b5081c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:41:55.859143 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a8ca24-2fe9-4852-b245-436403b5081c service nova] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:41:55.883946 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 701/1422] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:41:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:41:56.753553 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef186131-4b4d-4c16-80ad-4fcaf8bd5c65 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:56.754143 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef186131-4b4d-4c16-80ad-4fcaf8bd5c65 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:56.754143 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef186131-4b4d-4c16-80ad-4fcaf8bd5c65 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:56.755446 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 722/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 22:41:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:41:56.761237 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6a2c4-6227-492f-8219-140d89136b19 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:56.761670 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6a2c4-6227-492f-8219-140d89136b19 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:56.761766 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6a2c4-6227-492f-8219-140d89136b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:56.781572 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:56.838473 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-7aa6a2c4-6227-492f-8219-140d89136b19 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:41:56.839609 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-7aa6a2c4-6227-492f-8219-140d89136b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:56.937730 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 702/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 22:41:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:41:57.090244 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef186131-4b4d-4c16-80ad-4fcaf8bd5c65 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:41:57.091671 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-ef186131-4b4d-4c16-80ad-4fcaf8bd5c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:57.204975 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:57.241207 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef186131-4b4d-4c16-80ad-4fcaf8bd5c65 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:41:57.356108 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd5f0bf-6422-4bd3-be1d-46a037bf1eb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:41:57.356629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd5f0bf-6422-4bd3-be1d-46a037bf1eb9 service nova] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:57.356935 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd5f0bf-6422-4bd3-be1d-46a037bf1eb9 service nova] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:57.357208 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd5f0bf-6422-4bd3-be1d-46a037bf1eb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:57.358868 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-0bd5f0bf-6422-4bd3-be1d-46a037bf1eb9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:57.437122 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:57.451202 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd5f0bf-6422-4bd3-be1d-46a037bf1eb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:41:57.452816 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd5f0bf-6422-4bd3-be1d-46a037bf1eb9 service nova] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:41:57.470596 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 723/1425] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:41:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:41:58.361884 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5044659e-8794-46a1-b4d8-d27c9ea13215 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:58.362495 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5044659e-8794-46a1-b4d8-d27c9ea13215 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:58.362495 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5044659e-8794-46a1-b4d8-d27c9ea13215 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:58.363839 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 703/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 22:41:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:41:58.368924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed5b0a-ef32-49c9-a190-8365366fd6ce None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:58.369389 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed5b0a-ef32-49c9-a190-8365366fd6ce None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:58.369489 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed5b0a-ef32-49c9-a190-8365366fd6ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:58.448051 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-6fed5b0a-ef32-49c9-a190-8365366fd6ce None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:41:58.449549 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-6fed5b0a-ef32-49c9-a190-8365366fd6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:58.557028 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 724/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 22:41:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:41:58.717112 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5044659e-8794-46a1-b4d8-d27c9ea13215 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:41:58.718576 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-5044659e-8794-46a1-b4d8-d27c9ea13215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:58.802905 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:58.868296 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5044659e-8794-46a1-b4d8-d27c9ea13215 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:41:58.983878 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade33ce2-812b-4730-a286-cbf5e9177c10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:41:58.984418 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade33ce2-812b-4730-a286-cbf5e9177c10 service nova] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:58.984705 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade33ce2-812b-4730-a286-cbf5e9177c10 service nova] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:58.985006 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade33ce2-812b-4730-a286-cbf5e9177c10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:58.986599 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-ade33ce2-812b-4730-a286-cbf5e9177c10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:59.039874 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:59.084072 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade33ce2-812b-4730-a286-cbf5e9177c10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:41:59.085691 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade33ce2-812b-4730-a286-cbf5e9177c10 service nova] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:41:59.103407 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 704/1428] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:41:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:41:59.974317 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c49d5-a25f-4edd-8b6d-e275434c36c2 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:59.974900 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c49d5-a25f-4edd-8b6d-e275434c36c2 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:59.974900 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c49d5-a25f-4edd-8b6d-e275434c36c2 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:59.976323 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 725/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 22:41:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:41:59.981936 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6cee2c-8644-4787-8cc8-8fa297b8af73 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:59.982338 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6cee2c-8644-4787-8cc8-8fa297b8af73 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:59.982489 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6cee2c-8644-4787-8cc8-8fa297b8af73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:00.058745 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-5f6cee2c-8644-4787-8cc8-8fa297b8af73 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:42:00.059949 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-5f6cee2c-8644-4787-8cc8-8fa297b8af73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:00.159979 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 705/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 22:41:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:42:00.327248 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033c49d5-a25f-4edd-8b6d-e275434c36c2 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:42:00.328687 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-033c49d5-a25f-4edd-8b6d-e275434c36c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:00.377572 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:42:00.474563 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033c49d5-a25f-4edd-8b6d-e275434c36c2 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:42:00.605362 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75423e1b-d3c6-492b-b659-ecd17394bc99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:42:00.605883 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75423e1b-d3c6-492b-b659-ecd17394bc99 service nova] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:00.606159 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75423e1b-d3c6-492b-b659-ecd17394bc99 service nova] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:00.606427 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75423e1b-d3c6-492b-b659-ecd17394bc99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:00.608002 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-75423e1b-d3c6-492b-b659-ecd17394bc99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:00.615096 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:42:00.698149 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75423e1b-d3c6-492b-b659-ecd17394bc99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:42:00.699787 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75423e1b-d3c6-492b-b659-ecd17394bc99 service nova] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:42:00.717731 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 726/1431] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:42:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:42:01.604647 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6356b4-2a92-4802-a4fc-1709364c3d8b None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:01.605239 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6356b4-2a92-4802-a4fc-1709364c3d8b None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:01.605239 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6356b4-2a92-4802-a4fc-1709364c3d8b None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:01.606562 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 706/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 22:42:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:42:01.611599 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a54036-4133-4ca4-8dab-aed3abec4e70 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:01.612140 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a54036-4133-4ca4-8dab-aed3abec4e70 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:01.612260 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a54036-4133-4ca4-8dab-aed3abec4e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:01.689229 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-b4a54036-4133-4ca4-8dab-aed3abec4e70 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:42:01.690479 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-b4a54036-4133-4ca4-8dab-aed3abec4e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:01.757705 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:42:01.786446 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 727/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 22:42:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:42:01.937924 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6356b4-2a92-4802-a4fc-1709364c3d8b None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:42:01.939427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-8a6356b4-2a92-4802-a4fc-1709364c3d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:01.957188 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:42:02.085868 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6356b4-2a92-4802-a4fc-1709364c3d8b None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:42:02.173917 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:42:02.201147 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e85e669-0538-44d4-bb91-45039359b301 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:42:02.201779 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85e669-0538-44d4-bb91-45039359b301 service nova] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:02.202046 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85e669-0538-44d4-bb91-45039359b301 service nova] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:02.202319 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85e669-0538-44d4-bb91-45039359b301 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:02.203955 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-4e85e669-0538-44d4-bb91-45039359b301 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:02.296634 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e85e669-0538-44d4-bb91-45039359b301 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:42:02.298274 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e85e669-0538-44d4-bb91-45039359b301 service nova] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:42:02.316113 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 707/1434] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:42:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:42:03.301579 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf3871-e989-412d-8467-a4ed56f14bcc None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:03.302134 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf3871-e989-412d-8467-a4ed56f14bcc None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:03.302134 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf3871-e989-412d-8467-a4ed56f14bcc None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:03.303384 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 728/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 22:42:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:42:03.308637 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa41f3-d29d-4424-9e48-a6390c1681dc None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:03.309085 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa41f3-d29d-4424-9e48-a6390c1681dc None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:03.309201 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa41f3-d29d-4424-9e48-a6390c1681dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:03.385684 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-b8aa41f3-d29d-4424-9e48-a6390c1681dc None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:42:03.386882 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-b8aa41f3-d29d-4424-9e48-a6390c1681dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:03.429412 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:42:03.485265 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 708/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 22:42:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:42:03.653300 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bf3871-e989-412d-8467-a4ed56f14bcc None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:42:03.654729 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-f1bf3871-e989-412d-8467-a4ed56f14bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:03.798116 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bf3871-e989-412d-8467-a4ed56f14bcc None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:42:03.851848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:42:03.910902 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc8da7-f1a6-49ff-92ff-64a4c8b46ebc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:42:03.911430 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc8da7-f1a6-49ff-92ff-64a4c8b46ebc service nova] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:03.911708 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc8da7-f1a6-49ff-92ff-64a4c8b46ebc service nova] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:03.911977 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc8da7-f1a6-49ff-92ff-64a4c8b46ebc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:03.913586 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-86bc8da7-f1a6-49ff-92ff-64a4c8b46ebc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:04.007244 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bc8da7-f1a6-49ff-92ff-64a4c8b46ebc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:42:04.008862 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bc8da7-f1a6-49ff-92ff-64a4c8b46ebc service nova] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:42:04.027715 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 729/1437] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:42:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:42:04.902319 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd4056-d1e0-48ba-9bb1-e0d7756acd07 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:04.902864 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd4056-d1e0-48ba-9bb1-e0d7756acd07 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:04.902864 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd4056-d1e0-48ba-9bb1-e0d7756acd07 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:04.904130 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 709/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 22:42:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:42:04.909040 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4c6a7-45ef-40d9-9c61-c8904aa307e2 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:04.909436 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4c6a7-45ef-40d9-9c61-c8904aa307e2 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:04.909619 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4c6a7-45ef-40d9-9c61-c8904aa307e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:04.987661 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-76d4c6a7-45ef-40d9-9c61-c8904aa307e2 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:42:04.988914 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-76d4c6a7-45ef-40d9-9c61-c8904aa307e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:04.990945 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:42:05.085432 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 730/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 22:42:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:42:05.239108 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdd4056-d1e0-48ba-9bb1-e0d7756acd07 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:42:05.240621 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-fcdd4056-d1e0-48ba-9bb1-e0d7756acd07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:05.393398 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdd4056-d1e0-48ba-9bb1-e0d7756acd07 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:42:05.423218 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:42:05.511380 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5cf41f-d6b3-4639-8c8d-43b969e652ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:42:05.511879 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5cf41f-d6b3-4639-8c8d-43b969e652ef service nova] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:05.512159 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5cf41f-d6b3-4639-8c8d-43b969e652ef service nova] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:05.512440 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5cf41f-d6b3-4639-8c8d-43b969e652ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:05.514079 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-3d5cf41f-d6b3-4639-8c8d-43b969e652ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:05.611366 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5cf41f-d6b3-4639-8c8d-43b969e652ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:42:05.613000 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5cf41f-d6b3-4639-8c8d-43b969e652ef service nova] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:42:05.631480 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 710/1440] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:42:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:42:07.462589 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c358e15-079d-4bd2-989c-29cb9e2610f6 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:07.463121 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c358e15-079d-4bd2-989c-29cb9e2610f6 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:07.463121 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c358e15-079d-4bd2-989c-29cb9e2610f6 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:07.464506 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 731/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 22:42:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:42:07.469773 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c61469-60b3-4bcc-be8a-66afdffb57dd None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:07.470158 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c61469-60b3-4bcc-be8a-66afdffb57dd None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:07.474759 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c61469-60b3-4bcc-be8a-66afdffb57dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:07.513469 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:42:07.538718 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-29c61469-60b3-4bcc-be8a-66afdffb57dd None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:42:07.539920 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-29c61469-60b3-4bcc-be8a-66afdffb57dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:07.643496 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 711/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 22:42:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:42:07.924370 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c358e15-079d-4bd2-989c-29cb9e2610f6 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:42:07.925921 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-7c358e15-079d-4bd2-989c-29cb9e2610f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:08.030189 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c358e15-079d-4bd2-989c-29cb9e2610f6 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:42:08.063405 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:42:08.105706 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da941c0-1d70-4968-8d5b-88998d7f1f19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:42:08.106307 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da941c0-1d70-4968-8d5b-88998d7f1f19 service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:08.106581 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da941c0-1d70-4968-8d5b-88998d7f1f19 service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:08.106848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da941c0-1d70-4968-8d5b-88998d7f1f19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:08.108418 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-2da941c0-1d70-4968-8d5b-88998d7f1f19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:08.219962 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da941c0-1d70-4968-8d5b-88998d7f1f19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:42:08.221439 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da941c0-1d70-4968-8d5b-88998d7f1f19 service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:42:08.240674 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 732/1443] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:42:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:42:10.326091 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e006a7c-393b-4fad-8469-8779466c9b82 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:10.326718 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e006a7c-393b-4fad-8469-8779466c9b82 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:10.326718 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e006a7c-393b-4fad-8469-8779466c9b82 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:10.327947 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 712/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 22:42:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:42:10.333136 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c8682-c68c-435b-8bb3-829754109961 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:10.333549 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c8682-c68c-435b-8bb3-829754109961 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:10.333682 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c8682-c68c-435b-8bb3-829754109961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:10.399300 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-700c8682-c68c-435b-8bb3-829754109961 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:42:10.400507 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-700c8682-c68c-435b-8bb3-829754109961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:10.413259 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:42:10.498024 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 733/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 22:42:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:42:10.791054 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e006a7c-393b-4fad-8469-8779466c9b82 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:42:10.793231 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-6e006a7c-393b-4fad-8469-8779466c9b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:10.898653 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e006a7c-393b-4fad-8469-8779466c9b82 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:42:10.911025 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:42:10.975670 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1a125-d51e-4ec6-96b5-25349fbb12e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:42:10.976270 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1a125-d51e-4ec6-96b5-25349fbb12e3 service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:10.976547 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1a125-d51e-4ec6-96b5-25349fbb12e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:10.976847 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1a125-d51e-4ec6-96b5-25349fbb12e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:10.978425 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-7ea1a125-d51e-4ec6-96b5-25349fbb12e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:11.089533 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea1a125-d51e-4ec6-96b5-25349fbb12e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:42:11.090880 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea1a125-d51e-4ec6-96b5-25349fbb12e3 service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:42:11.109421 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 713/1446] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:42:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:42:13.496512 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc9afb-fb2c-4581-8fb5-282dca8a78db None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:13.497088 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc9afb-fb2c-4581-8fb5-282dca8a78db None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:13.497088 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc9afb-fb2c-4581-8fb5-282dca8a78db None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:13.498420 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 734/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 22:42:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:42:13.503674 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ced37c-896b-4111-b230-e2c49c696496 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:13.504072 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ced37c-896b-4111-b230-e2c49c696496 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:13.504227 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ced37c-896b-4111-b230-e2c49c696496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:13.555709 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:42:13.574406 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-21ced37c-896b-4111-b230-e2c49c696496 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:42:13.575615 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-21ced37c-896b-4111-b230-e2c49c696496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:13.674353 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 714/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 22:42:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:42:13.970378 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc9afb-fb2c-4581-8fb5-282dca8a78db None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:42:13.971987 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-bbbc9afb-fb2c-4581-8fb5-282dca8a78db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:14.083056 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc9afb-fb2c-4581-8fb5-282dca8a78db None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:42:14.125344 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:42:14.158809 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9666157d-e31a-4973-b21f-2797570fb125 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:42:14.159412 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9666157d-e31a-4973-b21f-2797570fb125 service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:14.159692 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9666157d-e31a-4973-b21f-2797570fb125 service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:14.159964 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9666157d-e31a-4973-b21f-2797570fb125 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:14.161595 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-9666157d-e31a-4973-b21f-2797570fb125 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:14.273500 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9666157d-e31a-4973-b21f-2797570fb125 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:42:14.274860 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9666157d-e31a-4973-b21f-2797570fb125 service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:42:14.294610 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 735/1449] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:42:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:42:16.396669 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12816e76-5c24-4ead-bb6b-662ef0581f7a None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:16.397233 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12816e76-5c24-4ead-bb6b-662ef0581f7a None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:16.397233 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12816e76-5c24-4ead-bb6b-662ef0581f7a None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:16.398569 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 715/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 22:42:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:42:16.403801 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5491b-152e-45b3-a54d-5f0148be2fd1 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:16.410071 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5491b-152e-45b3-a54d-5f0148be2fd1 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:16.410071 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5491b-152e-45b3-a54d-5f0148be2fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:16.472454 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-f2c5491b-152e-45b3-a54d-5f0148be2fd1 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:42:16.473690 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-f2c5491b-152e-45b3-a54d-5f0148be2fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:16.495229 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:42:16.573791 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 736/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 22:42:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:42:16.862692 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12816e76-5c24-4ead-bb6b-662ef0581f7a None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:42:16.864237 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-12816e76-5c24-4ead-bb6b-662ef0581f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:16.971787 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12816e76-5c24-4ead-bb6b-662ef0581f7a None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:42:16.992151 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:42:17.047455 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cafe80-8303-4e9c-8d02-9e29d8d0d104 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:42:17.048059 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cafe80-8303-4e9c-8d02-9e29d8d0d104 service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:17.048342 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cafe80-8303-4e9c-8d02-9e29d8d0d104 service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:17.048624 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cafe80-8303-4e9c-8d02-9e29d8d0d104 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:17.050262 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-a9cafe80-8303-4e9c-8d02-9e29d8d0d104 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:17.162503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cafe80-8303-4e9c-8d02-9e29d8d0d104 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:42:17.163850 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cafe80-8303-4e9c-8d02-9e29d8d0d104 service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:42:17.183470 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 716/1452] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:42:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:42:19.223608 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77194a38-af6e-402f-8a5f-aede9c625698 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:19.224190 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77194a38-af6e-402f-8a5f-aede9c625698 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:19.224190 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77194a38-af6e-402f-8a5f-aede9c625698 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:19.225614 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 737/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 22:42:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:42:19.231485 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e9318-4a3b-42be-91de-78242b8c5140 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:19.231888 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e9318-4a3b-42be-91de-78242b8c5140 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:19.232030 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e9318-4a3b-42be-91de-78242b8c5140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:19.295647 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:42:19.298435 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-eb0e9318-4a3b-42be-91de-78242b8c5140 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:42:19.299699 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-eb0e9318-4a3b-42be-91de-78242b8c5140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:19.398768 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 717/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 22:42:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:42:19.685870 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77194a38-af6e-402f-8a5f-aede9c625698 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:42:19.687405 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-77194a38-af6e-402f-8a5f-aede9c625698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:19.798699 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77194a38-af6e-402f-8a5f-aede9c625698 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:42:19.858001 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:42:19.883245 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6ca468-c52d-46d9-a73c-f2ae636b555c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:42:19.883850 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ca468-c52d-46d9-a73c-f2ae636b555c service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:19.884131 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ca468-c52d-46d9-a73c-f2ae636b555c service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:19.884406 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ca468-c52d-46d9-a73c-f2ae636b555c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:19.886765 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-db6ca468-c52d-46d9-a73c-f2ae636b555c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:19.994210 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6ca468-c52d-46d9-a73c-f2ae636b555c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:42:19.996268 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6ca468-c52d-46d9-a73c-f2ae636b555c service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:42:20.014802 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 738/1455] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:42:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:42:22.003510 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33105007-6643-41f9-a049-250f322060a6 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:22.004367 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33105007-6643-41f9-a049-250f322060a6 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:22.004367 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33105007-6643-41f9-a049-250f322060a6 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:22.005959 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 718/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 22:42:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:42:22.011580 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9a25b-56ba-4bf7-91de-2108acb41fdd None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:22.012100 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9a25b-56ba-4bf7-91de-2108acb41fdd None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:22.012100 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9a25b-56ba-4bf7-91de-2108acb41fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:22.082387 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-18f9a25b-56ba-4bf7-91de-2108acb41fdd None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:42:22.083601 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-18f9a25b-56ba-4bf7-91de-2108acb41fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:22.113776 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:42:22.183393 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 739/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 22:42:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:42:22.471236 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33105007-6643-41f9-a049-250f322060a6 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:42:22.473006 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-33105007-6643-41f9-a049-250f322060a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:22.580476 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33105007-6643-41f9-a049-250f322060a6 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:42:22.618251 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:42:22.664365 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23770ea-dee4-4c55-9dea-d81ec82f20b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:42:22.665030 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23770ea-dee4-4c55-9dea-d81ec82f20b1 service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:22.665310 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23770ea-dee4-4c55-9dea-d81ec82f20b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:22.665583 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23770ea-dee4-4c55-9dea-d81ec82f20b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:22.667811 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-f23770ea-dee4-4c55-9dea-d81ec82f20b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:22.777045 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23770ea-dee4-4c55-9dea-d81ec82f20b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:42:22.779134 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23770ea-dee4-4c55-9dea-d81ec82f20b1 service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:42:22.797624 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 719/1458] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:42:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:42:24.783355 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3762fe-f32b-4416-8d85-f0b104087e4c None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:24.783909 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3762fe-f32b-4416-8d85-f0b104087e4c None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:24.783909 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3762fe-f32b-4416-8d85-f0b104087e4c None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:24.785260 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 740/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 22:42:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:42:24.790756 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e44b634-0f95-4f33-ac88-821fe29bc810 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:24.791204 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e44b634-0f95-4f33-ac88-821fe29bc810 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:24.791829 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e44b634-0f95-4f33-ac88-821fe29bc810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:24.858488 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-8e44b634-0f95-4f33-ac88-821fe29bc810 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:42:24.859677 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-8e44b634-0f95-4f33-ac88-821fe29bc810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:24.867965 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:42:24.956984 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 720/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 22:42:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:42:25.243580 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3762fe-f32b-4416-8d85-f0b104087e4c None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:42:25.248110 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-0c3762fe-f32b-4416-8d85-f0b104087e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:25.355870 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3762fe-f32b-4416-8d85-f0b104087e4c None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:42:25.410913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:42:25.429457 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d90da6-1f50-488c-b635-9c99a9193647 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:42:25.430067 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d90da6-1f50-488c-b635-9c99a9193647 service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:25.430347 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d90da6-1f50-488c-b635-9c99a9193647 service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:25.430619 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d90da6-1f50-488c-b635-9c99a9193647 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:25.432249 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-63d90da6-1f50-488c-b635-9c99a9193647 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:25.544179 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d90da6-1f50-488c-b635-9c99a9193647 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:42:25.545648 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d90da6-1f50-488c-b635-9c99a9193647 service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:42:25.563916 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 741/1461] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:42:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:42:27.546714 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce048ad-8384-41a3-96ab-3d59d7dc9722 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:27.547192 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce048ad-8384-41a3-96ab-3d59d7dc9722 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:27.547292 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce048ad-8384-41a3-96ab-3d59d7dc9722 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:27.548583 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 721/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 22:42:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:42:27.553912 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5a0df-b599-4b50-b813-a1f0b98076e0 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:27.554358 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5a0df-b599-4b50-b813-a1f0b98076e0 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:27.554493 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5a0df-b599-4b50-b813-a1f0b98076e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:27.622079 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-f7b5a0df-b599-4b50-b813-a1f0b98076e0 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:42:27.623290 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-f7b5a0df-b599-4b50-b813-a1f0b98076e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:27.662284 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:42:27.720438 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 742/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 22:42:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:42:28.006221 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce048ad-8384-41a3-96ab-3d59d7dc9722 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:42:28.007768 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-0ce048ad-8384-41a3-96ab-3d59d7dc9722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:28.111464 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce048ad-8384-41a3-96ab-3d59d7dc9722 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:42:28.148350 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:42:28.186742 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073f5083-8204-4ef0-911d-dff0fd07e03d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:42:28.187351 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f5083-8204-4ef0-911d-dff0fd07e03d service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:28.187633 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f5083-8204-4ef0-911d-dff0fd07e03d service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:28.187904 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f5083-8204-4ef0-911d-dff0fd07e03d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:28.189582 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-073f5083-8204-4ef0-911d-dff0fd07e03d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:28.300241 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073f5083-8204-4ef0-911d-dff0fd07e03d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:42:28.301620 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073f5083-8204-4ef0-911d-dff0fd07e03d service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:42:28.320483 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 722/1464] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:42:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:42:30.327686 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf00581-74de-490a-9fc0-25037a0ef9e9 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:30.328289 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf00581-74de-490a-9fc0-25037a0ef9e9 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:30.328289 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf00581-74de-490a-9fc0-25037a0ef9e9 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:30.329655 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 743/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 22:42:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:42:30.335159 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8cced5-0964-421d-93b7-acf0f3c79cff None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:30.335645 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8cced5-0964-421d-93b7-acf0f3c79cff None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:30.335751 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8cced5-0964-421d-93b7-acf0f3c79cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:30.417054 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-2c8cced5-0964-421d-93b7-acf0f3c79cff None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:42:30.418465 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-2c8cced5-0964-421d-93b7-acf0f3c79cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:30.444066 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:42:30.530860 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 723/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 22:42:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:42:30.818705 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf00581-74de-490a-9fc0-25037a0ef9e9 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:42:30.820209 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-cbf00581-74de-490a-9fc0-25037a0ef9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:30.930948 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf00581-74de-490a-9fc0-25037a0ef9e9 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:42:30.999254 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:42:31.006065 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6b86a2-7ce4-46a8-ba9a-2bd94b2e544d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:42:31.006668 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b86a2-7ce4-46a8-ba9a-2bd94b2e544d service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:31.006945 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b86a2-7ce4-46a8-ba9a-2bd94b2e544d service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:31.007212 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b86a2-7ce4-46a8-ba9a-2bd94b2e544d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:31.008829 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-7a6b86a2-7ce4-46a8-ba9a-2bd94b2e544d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:31.118950 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6b86a2-7ce4-46a8-ba9a-2bd94b2e544d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:42:31.121297 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6b86a2-7ce4-46a8-ba9a-2bd94b2e544d service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:42:31.140050 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 744/1467] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:42:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:42:33.085950 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21359be-3816-4201-a4e9-85cbe0af5e3f None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:33.086582 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21359be-3816-4201-a4e9-85cbe0af5e3f None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:33.086870 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21359be-3816-4201-a4e9-85cbe0af5e3f None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:33.088323 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 724/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 22:42:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:42:33.093767 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463642e-9383-4cdf-ac1a-df755fd94c64 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:33.094193 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463642e-9383-4cdf-ac1a-df755fd94c64 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:33.094325 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463642e-9383-4cdf-ac1a-df755fd94c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:33.159934 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-0463642e-9383-4cdf-ac1a-df755fd94c64 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:42:33.161179 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-0463642e-9383-4cdf-ac1a-df755fd94c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:33.211006 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:42:33.262108 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 745/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 22:42:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:42:33.544571 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21359be-3816-4201-a4e9-85cbe0af5e3f None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:42:33.546155 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-e21359be-3816-4201-a4e9-85cbe0af5e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:33.654857 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21359be-3816-4201-a4e9-85cbe0af5e3f None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:42:33.702735 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:42:33.728846 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c30c3ce-7249-4a86-ac01-1979fa1cc99d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:42:33.729618 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c30c3ce-7249-4a86-ac01-1979fa1cc99d service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:33.729896 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c30c3ce-7249-4a86-ac01-1979fa1cc99d service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:33.730165 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c30c3ce-7249-4a86-ac01-1979fa1cc99d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:33.732410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-7c30c3ce-7249-4a86-ac01-1979fa1cc99d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:33.841300 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c30c3ce-7249-4a86-ac01-1979fa1cc99d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:42:33.842662 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c30c3ce-7249-4a86-ac01-1979fa1cc99d service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:42:33.861646 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 725/1470] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:42:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:42:43.260289 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=22 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:42:43.859395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=66 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:42:54.453921 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e294c40-59bb-423e-bce2-2fe2536551de None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:54.454488 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e294c40-59bb-423e-bce2-2fe2536551de None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:54.454488 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e294c40-59bb-423e-bce2-2fe2536551de None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:54.455823 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 746/1471] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:42:54.461138 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df42a31-dd5b-4f0c-87c2-3d36911f2915 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:54.461583 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df42a31-dd5b-4f0c-87c2-3d36911f2915 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:54.461691 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df42a31-dd5b-4f0c-87c2-3d36911f2915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:54.527637 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-2df42a31-dd5b-4f0c-87c2-3d36911f2915 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:42:54.528913 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-2df42a31-dd5b-4f0c-87c2-3d36911f2915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:54.633148 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 726/1472] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:42:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:42:54.933952 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca7028-8499-45de-a2ed-066ddf70397f None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:54.934237 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca7028-8499-45de-a2ed-066ddf70397f None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:54.934511 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca7028-8499-45de-a2ed-066ddf70397f None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:54.935906 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 747/1473] 10.222.0.52 () {54 vars in 967 bytes} [Mon Aug 29 22:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:42:54.941498 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5619cd-3518-4129-8bbf-a68ba7860b8f None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:54.941787 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5619cd-3518-4129-8bbf-a68ba7860b8f None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:54.942067 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5619cd-3518-4129-8bbf-a68ba7860b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:55.020303 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-0f5619cd-3518-4129-8bbf-a68ba7860b8f None None] MFA Rules not processed for user `9c2569e7cd774199aa5750fd6a850060`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:42:55.021537 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-0f5619cd-3518-4129-8bbf-a68ba7860b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:55.042603 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:42:55.099409 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 727/1474] 10.222.0.52 () {58 vars in 1064 bytes} [Mon Aug 29 22:42:54 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:42:55.107754 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca7028-8499-45de-a2ed-066ddf70397f None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:42:55.109308 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-bdca7028-8499-45de-a2ed-066ddf70397f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:55.216706 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca7028-8499-45de-a2ed-066ddf70397f None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:42:55.291810 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd1f2f-52c3-466c-8194-29d274662ea6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c2569e7cd774199aa5750fd6a850060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:42:55.292329 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd1f2f-52c3-466c-8194-29d274662ea6 service glance] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:55.292604 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd1f2f-52c3-466c-8194-29d274662ea6 service glance] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:55.292911 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd1f2f-52c3-466c-8194-29d274662ea6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:55.294488 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-8bfd1f2f-52c3-466c-8194-29d274662ea6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:55.347995 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:42:55.404102 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfd1f2f-52c3-466c-8194-29d274662ea6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:42:55.405583 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfd1f2f-52c3-466c-8194-29d274662ea6 service glance] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:42:55.424071 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 748/1475] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:42:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:42:57.463481 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccdefa0-c8a8-4f9f-8304-148e4d45e5cf None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:57.464237 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccdefa0-c8a8-4f9f-8304-148e4d45e5cf None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:57.464237 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccdefa0-c8a8-4f9f-8304-148e4d45e5cf None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:57.465508 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 728/1476] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:42:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:42:57.479536 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f20990-1259-42eb-8734-4614f81be1ce None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:57.479536 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f20990-1259-42eb-8734-4614f81be1ce None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:57.479536 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f20990-1259-42eb-8734-4614f81be1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:57.543722 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-85f20990-1259-42eb-8734-4614f81be1ce None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:42:57.551551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-85f20990-1259-42eb-8734-4614f81be1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:57.645404 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 749/1477] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:42:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:42:57.915323 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccdefa0-c8a8-4f9f-8304-148e4d45e5cf None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:42:57.917174 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-fccdefa0-c8a8-4f9f-8304-148e4d45e5cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:58.044458 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccdefa0-c8a8-4f9f-8304-148e4d45e5cf None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:42:58.095414 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:42:58.142283 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc70d5ce-ce4c-4fc8-87b2-8a3d893966f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:42:58.142887 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc70d5ce-ce4c-4fc8-87b2-8a3d893966f6 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:58.150162 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc70d5ce-ce4c-4fc8-87b2-8a3d893966f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:58.150162 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc70d5ce-ce4c-4fc8-87b2-8a3d893966f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:58.150162 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc70d5ce-ce4c-4fc8-87b2-8a3d893966f6 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:42:58.150162 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: WARNING py.warnings [None req-cc70d5ce-ce4c-4fc8-87b2-8a3d893966f6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 22:42:58.150162 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: warnings.warn(msg) Aug 29 22:42:58.150162 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: Aug 29 22:42:58.150162 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc70d5ce-ce4c-4fc8-87b2-8a3d893966f6 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:42:58.166765 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 729/1478] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Aug 29 22:42:57 2022] GET /identity/v3/projects => generated 2553 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:43:00.101001 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0214eb-509e-49d1-96bd-6d0978df1fe5 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:00.101568 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0214eb-509e-49d1-96bd-6d0978df1fe5 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:00.101568 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0214eb-509e-49d1-96bd-6d0978df1fe5 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:00.102891 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 750/1479] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:43:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:43:00.108225 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3b131-3068-46b3-99d9-8d3e5fffa169 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:00.108623 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3b131-3068-46b3-99d9-8d3e5fffa169 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:00.108806 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3b131-3068-46b3-99d9-8d3e5fffa169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:00.180850 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-59a3b131-3068-46b3-99d9-8d3e5fffa169 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:43:00.182168 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-59a3b131-3068-46b3-99d9-8d3e5fffa169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:00.284077 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 730/1480] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:43:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:43:00.597048 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0214eb-509e-49d1-96bd-6d0978df1fe5 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:00.597048 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-cf0214eb-509e-49d1-96bd-6d0978df1fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:00.605285 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:43:00.748561 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0214eb-509e-49d1-96bd-6d0978df1fe5 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:00.828871 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:43:00.862193 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c120c03-4f3e-477f-a2f0-203fdcc6eb5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:00.862795 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c120c03-4f3e-477f-a2f0-203fdcc6eb5a service nova] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:00.863079 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c120c03-4f3e-477f-a2f0-203fdcc6eb5a service nova] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:00.863350 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c120c03-4f3e-477f-a2f0-203fdcc6eb5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:00.865063 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-7c120c03-4f3e-477f-a2f0-203fdcc6eb5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:00.957339 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c120c03-4f3e-477f-a2f0-203fdcc6eb5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:00.958919 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c120c03-4f3e-477f-a2f0-203fdcc6eb5a service nova] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:00.977524 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 751/1481] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:43:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:43:03.108180 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3606b456-543b-488d-bb22-440bd4657cc2 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:03.108757 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3606b456-543b-488d-bb22-440bd4657cc2 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:03.108757 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3606b456-543b-488d-bb22-440bd4657cc2 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:03.110169 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 731/1482] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:43:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:43:03.115793 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eff728-bf5c-4d8f-a163-328f205c1463 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:03.116275 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eff728-bf5c-4d8f-a163-328f205c1463 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:03.116389 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eff728-bf5c-4d8f-a163-328f205c1463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:03.181437 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-c6eff728-bf5c-4d8f-a163-328f205c1463 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:43:03.183242 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-c6eff728-bf5c-4d8f-a163-328f205c1463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:03.230085 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:43:03.280072 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 752/1483] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:43:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:43:03.575797 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3606b456-543b-488d-bb22-440bd4657cc2 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:03.577424 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-3606b456-543b-488d-bb22-440bd4657cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:03.601176 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:43:03.681270 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3606b456-543b-488d-bb22-440bd4657cc2 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:03.754194 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391c4453-280a-47fe-adbf-ed15a592d5b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:03.754803 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c4453-280a-47fe-adbf-ed15a592d5b1 service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:03.755119 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c4453-280a-47fe-adbf-ed15a592d5b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:03.755389 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c4453-280a-47fe-adbf-ed15a592d5b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:03.757051 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-391c4453-280a-47fe-adbf-ed15a592d5b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:03.833346 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:43:03.867891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391c4453-280a-47fe-adbf-ed15a592d5b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:03.869782 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391c4453-280a-47fe-adbf-ed15a592d5b1 service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:03.889526 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 732/1484] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:43:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:43:05.889581 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b517ba-ee57-499f-8c3d-690deec93df2 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:05.890114 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b517ba-ee57-499f-8c3d-690deec93df2 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:05.890114 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b517ba-ee57-499f-8c3d-690deec93df2 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:05.891387 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 753/1485] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:43:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:43:05.896451 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e340c-a9d4-4036-b6af-7f775811d76d None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:05.896924 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e340c-a9d4-4036-b6af-7f775811d76d None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:05.897028 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e340c-a9d4-4036-b6af-7f775811d76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:05.962137 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-2e5e340c-a9d4-4036-b6af-7f775811d76d None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:43:05.963880 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-2e5e340c-a9d4-4036-b6af-7f775811d76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:06.065052 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 733/1486] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:43:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:43:06.360060 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b517ba-ee57-499f-8c3d-690deec93df2 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:06.361659 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-d6b517ba-ee57-499f-8c3d-690deec93df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:06.479452 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b517ba-ee57-499f-8c3d-690deec93df2 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:06.553431 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:43:06.555216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d80db60-7470-4d2c-83bb-2a34466f9e10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:06.555824 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d80db60-7470-4d2c-83bb-2a34466f9e10 service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:06.556105 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d80db60-7470-4d2c-83bb-2a34466f9e10 service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:06.556379 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d80db60-7470-4d2c-83bb-2a34466f9e10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:06.558619 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-1d80db60-7470-4d2c-83bb-2a34466f9e10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:06.676139 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d80db60-7470-4d2c-83bb-2a34466f9e10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:06.677505 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d80db60-7470-4d2c-83bb-2a34466f9e10 service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:06.696525 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 754/1487] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:43:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:43:08.795103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083d18b-d040-4187-b0df-83be93826f08 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:08.795627 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083d18b-d040-4187-b0df-83be93826f08 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:08.795627 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083d18b-d040-4187-b0df-83be93826f08 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:08.796943 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 734/1488] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:43:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:43:08.802271 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfce7b7-6930-4bc2-8ffa-0439d024c750 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:08.802729 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfce7b7-6930-4bc2-8ffa-0439d024c750 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:08.802837 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfce7b7-6930-4bc2-8ffa-0439d024c750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:08.868611 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-cbfce7b7-6930-4bc2-8ffa-0439d024c750 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:43:08.870452 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-cbfce7b7-6930-4bc2-8ffa-0439d024c750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:08.923868 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:43:08.969321 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 755/1489] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:43:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:43:09.266483 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b083d18b-d040-4187-b0df-83be93826f08 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:09.268029 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-b083d18b-d040-4187-b0df-83be93826f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:09.304419 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:43:09.373179 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b083d18b-d040-4187-b0df-83be93826f08 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:09.448861 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62007b7b-3d05-46a4-b517-2adc2d5afac8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:09.449470 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62007b7b-3d05-46a4-b517-2adc2d5afac8 service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:09.449752 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62007b7b-3d05-46a4-b517-2adc2d5afac8 service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:09.450032 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62007b7b-3d05-46a4-b517-2adc2d5afac8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:09.451634 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-62007b7b-3d05-46a4-b517-2adc2d5afac8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:09.532752 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:43:09.563750 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62007b7b-3d05-46a4-b517-2adc2d5afac8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:09.565245 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62007b7b-3d05-46a4-b517-2adc2d5afac8 service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:09.584181 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 735/1490] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:43:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:43:11.973025 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23efc1c-6f2e-4f4d-9c61-5e7408396e5f None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:11.973557 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23efc1c-6f2e-4f4d-9c61-5e7408396e5f None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:11.973673 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23efc1c-6f2e-4f4d-9c61-5e7408396e5f None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:11.974968 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 756/1491] 10.222.0.52 () {54 vars in 927 bytes} [Mon Aug 29 22:43:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:43:11.981123 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadbaf9c-a153-4c67-8d5f-98fb5f63dc33 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:11.981549 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadbaf9c-a153-4c67-8d5f-98fb5f63dc33 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:11.981663 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadbaf9c-a153-4c67-8d5f-98fb5f63dc33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:12.048890 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-fadbaf9c-a153-4c67-8d5f-98fb5f63dc33 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:43:12.051068 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-fadbaf9c-a153-4c67-8d5f-98fb5f63dc33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:12.156308 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 736/1492] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Aug 29 22:43:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:43:12.442913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23efc1c-6f2e-4f4d-9c61-5e7408396e5f None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:12.444730 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-a23efc1c-6f2e-4f4d-9c61-5e7408396e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:12.562233 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23efc1c-6f2e-4f4d-9c61-5e7408396e5f None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:12.639404 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc662fa2-5d10-4076-85f4-71c257f0192a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:12.640010 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc662fa2-5d10-4076-85f4-71c257f0192a service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:12.640294 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc662fa2-5d10-4076-85f4-71c257f0192a service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:12.640569 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc662fa2-5d10-4076-85f4-71c257f0192a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:12.642337 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-bc662fa2-5d10-4076-85f4-71c257f0192a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:12.651735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:43:12.757874 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc662fa2-5d10-4076-85f4-71c257f0192a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:12.759301 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc662fa2-5d10-4076-85f4-71c257f0192a service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:12.777427 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 757/1493] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:43:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:43:22.154496 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=82 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:43:22.774985 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=50 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:36.264076 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145d0a1-a3d3-4016-b17d-2acdb597c874 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:36.264686 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145d0a1-a3d3-4016-b17d-2acdb597c874 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:36.264686 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145d0a1-a3d3-4016-b17d-2acdb597c874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:36.343244 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-a145d0a1-a3d3-4016-b17d-2acdb597c874 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:36.344473 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-a145d0a1-a3d3-4016-b17d-2acdb597c874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:36.445051 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 737/1494] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:44:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:36.453553 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e48981-8ee0-4d89-8883-fd8595381848 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:36.454092 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e48981-8ee0-4d89-8883-fd8595381848 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:36.454092 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e48981-8ee0-4d89-8883-fd8595381848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:36.529885 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-11e48981-8ee0-4d89-8883-fd8595381848 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:36.531033 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-11e48981-8ee0-4d89-8883-fd8595381848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:36.629239 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 758/1495] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:44:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:36.635645 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145d0a1-a3d3-4016-b17d-2acdb597c874 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:36.635645 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-a145d0a1-a3d3-4016-b17d-2acdb597c874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:36.647261 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145d0a1-a3d3-4016-b17d-2acdb597c874 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:36.716575 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:36.751660 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27e35a-eb23-44f8-8784-bb65ca152963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:36.752273 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27e35a-eb23-44f8-8784-bb65ca152963 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:36.752577 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27e35a-eb23-44f8-8784-bb65ca152963 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:36.752875 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27e35a-eb23-44f8-8784-bb65ca152963 admin admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:36.753519 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb27e35a-eb23-44f8-8784-bb65ca152963 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:44:36.753820 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb27e35a-eb23-44f8-8784-bb65ca152963 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:36.754879 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: WARNING py.warnings [None req-cb27e35a-eb23-44f8-8784-bb65ca152963 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 29 22:44:36.754879 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: warnings.warn(msg) Aug 29 22:44:36.754879 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: Aug 29 22:44:36.755782 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb27e35a-eb23-44f8-8784-bb65ca152963 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:36.767572 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 738/1496] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:44:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:44:36.776345 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e48981-8ee0-4d89-8883-fd8595381848 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:36.786547 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e48981-8ee0-4d89-8883-fd8595381848 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:36.851917 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:36.885720 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c84f83-da1a-4031-9e16-3cafa469a655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:36.886328 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c84f83-da1a-4031-9e16-3cafa469a655 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:36.886610 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c84f83-da1a-4031-9e16-3cafa469a655 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:36.886887 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c84f83-da1a-4031-9e16-3cafa469a655 admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:36.887517 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c84f83-da1a-4031-9e16-3cafa469a655 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:36.890223 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: WARNING py.warnings [None req-b8c84f83-da1a-4031-9e16-3cafa469a655 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 22:44:36.890223 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: warnings.warn(msg) Aug 29 22:44:36.890223 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: Aug 29 22:44:36.891430 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c84f83-da1a-4031-9e16-3cafa469a655 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:37.079464 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 759/1497] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:44:36 2022] POST /identity/v3/projects => generated 377 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:44:37.083846 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27e35a-eb23-44f8-8784-bb65ca152963 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:37.085409 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-cb27e35a-eb23-44f8-8784-bb65ca152963 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:37.119091 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27e35a-eb23-44f8-8784-bb65ca152963 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:37.214423 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72653619-d1c2-4df0-bde7-6c2b6d42801d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:37.215022 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72653619-d1c2-4df0-bde7-6c2b6d42801d admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:37.215298 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72653619-d1c2-4df0-bde7-6c2b6d42801d admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:37.215565 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72653619-d1c2-4df0-bde7-6c2b6d42801d admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:37.216185 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72653619-d1c2-4df0-bde7-6c2b6d42801d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:37.217324 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: WARNING py.warnings [None req-72653619-d1c2-4df0-bde7-6c2b6d42801d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 22:44:37.217324 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: warnings.warn(msg) Aug 29 22:44:37.217324 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: Aug 29 22:44:37.218523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72653619-d1c2-4df0-bde7-6c2b6d42801d admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:37.393451 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 739/1498] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:44:37 2022] POST /identity/v3/users => generated 289 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:44:37.397567 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c84f83-da1a-4031-9e16-3cafa469a655 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:37.399108 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-b8c84f83-da1a-4031-9e16-3cafa469a655 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:37.462499 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c84f83-da1a-4031-9e16-3cafa469a655 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:37.551094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:37.563072 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb96ecb-4a9c-4ce6-b883-4e20321dd890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:37.564329 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb96ecb-4a9c-4ce6-b883-4e20321dd890 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:37.564329 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb96ecb-4a9c-4ce6-b883-4e20321dd890 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:37.564329 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb96ecb-4a9c-4ce6-b883-4e20321dd890 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:37.566160 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb96ecb-4a9c-4ce6-b883-4e20321dd890 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:37.566160 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: WARNING py.warnings [None req-cdb96ecb-4a9c-4ce6-b883-4e20321dd890 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 29 22:44:37.566160 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: warnings.warn(msg) Aug 29 22:44:37.566160 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: Aug 29 22:44:37.566730 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb96ecb-4a9c-4ce6-b883-4e20321dd890 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:37.580158 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 760/1499] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:44:37 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:44:37.584268 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72653619-d1c2-4df0-bde7-6c2b6d42801d admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:37.586089 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-72653619-d1c2-4df0-bde7-6c2b6d42801d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:37.598373 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72653619-d1c2-4df0-bde7-6c2b6d42801d admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:37.624556 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:37.695208 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b960569-0e44-4e6f-806c-6f92a21096a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:37.696067 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b960569-0e44-4e6f-806c-6f92a21096a8 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:37.699140 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b960569-0e44-4e6f-806c-6f92a21096a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:37.699140 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b960569-0e44-4e6f-806c-6f92a21096a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:37.699140 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b960569-0e44-4e6f-806c-6f92a21096a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:37.699140 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: WARNING py.warnings [None req-3b960569-0e44-4e6f-806c-6f92a21096a8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 29 22:44:37.699140 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: warnings.warn(msg) Aug 29 22:44:37.699140 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: Aug 29 22:44:37.699654 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b960569-0e44-4e6f-806c-6f92a21096a8 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:37.711609 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 740/1500] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:44:37 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:44:37.715994 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb96ecb-4a9c-4ce6-b883-4e20321dd890 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:37.718258 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-cdb96ecb-4a9c-4ce6-b883-4e20321dd890 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:37.755481 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb96ecb-4a9c-4ce6-b883-4e20321dd890 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:37.849792 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6201086c-82a8-48b9-b57b-808bfccbb501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:37.850454 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6201086c-82a8-48b9-b57b-808bfccbb501 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:37.850735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6201086c-82a8-48b9-b57b-808bfccbb501 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:37.851007 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6201086c-82a8-48b9-b57b-808bfccbb501 admin admin] PATH_INFO: `/v3/projects/2ab7617cd5d54caea5e6b4fd2cabdae6/users/636291677b504c17b49faa2b08bb32b1/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:37.873586 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6201086c-82a8-48b9-b57b-808bfccbb501 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ab7617cd5d54caea5e6b4fd2cabdae6, user_id=636291677b504c17b49faa2b08bb32b1, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:37.874605 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: WARNING py.warnings [None req-6201086c-82a8-48b9-b57b-808bfccbb501 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 22:44:37.874605 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: warnings.warn(msg) Aug 29 22:44:37.874605 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: Aug 29 22:44:37.876565 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6201086c-82a8-48b9-b57b-808bfccbb501 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:37.961438 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 761/1501] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:44:37 2022] PUT /identity/v3/projects/2ab7617cd5d54caea5e6b4fd2cabdae6/users/636291677b504c17b49faa2b08bb32b1/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:37.974842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccca0ae-de40-4fda-b1fa-ea854ecf5791 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:37.975131 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccca0ae-de40-4fda-b1fa-ea854ecf5791 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:37.975408 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccca0ae-de40-4fda-b1fa-ea854ecf5791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:38.034373 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-8ccca0ae-de40-4fda-b1fa-ea854ecf5791 None None] MFA Rules not processed for user `636291677b504c17b49faa2b08bb32b1`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:38.035584 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-8ccca0ae-de40-4fda-b1fa-ea854ecf5791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:38.116049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 741/1502] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:44:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:38.129539 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824fd41-f00b-49dd-8cd6-198cc3e1edbc None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:38.129827 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824fd41-f00b-49dd-8cd6-198cc3e1edbc None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:38.130123 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824fd41-f00b-49dd-8cd6-198cc3e1edbc None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:38.131493 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 762/1503] 10.222.0.52 () {54 vars in 964 bytes} [Mon Aug 29 22:44:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:44:38.136339 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b6401-0693-49ac-9f16-76281250a44a None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:38.136651 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b6401-0693-49ac-9f16-76281250a44a None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:38.136957 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b6401-0693-49ac-9f16-76281250a44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:38.170556 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:38.215901 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-a29b6401-0693-49ac-9f16-76281250a44a None None] MFA Rules not processed for user `6e782c8acaaa42c4bd78ddc7c8cc654b`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:38.217737 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-a29b6401-0693-49ac-9f16-76281250a44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:38.295234 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 742/1504] 10.222.0.52 () {58 vars in 1061 bytes} [Mon Aug 29 22:44:38 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:38.303058 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a824fd41-f00b-49dd-8cd6-198cc3e1edbc None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:38.304530 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-a824fd41-f00b-49dd-8cd6-198cc3e1edbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:38.358015 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:38.415676 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a824fd41-f00b-49dd-8cd6-198cc3e1edbc None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:38.496236 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6325660-45e6-4de4-959b-208cb549000a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:38.496908 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6325660-45e6-4de4-959b-208cb549000a service cinder] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:38.497100 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6325660-45e6-4de4-959b-208cb549000a service cinder] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:38.497347 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6325660-45e6-4de4-959b-208cb549000a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:38.499125 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-a6325660-45e6-4de4-959b-208cb549000a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:38.605201 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6325660-45e6-4de4-959b-208cb549000a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:38.606720 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6325660-45e6-4de4-959b-208cb549000a service cinder] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:38.612075 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:38.629942 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 763/1505] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:44:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:44:38.679336 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82858007-0f10-4846-ba07-743944445d5d None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:38.679765 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82858007-0f10-4846-ba07-743944445d5d None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:38.679890 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82858007-0f10-4846-ba07-743944445d5d None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:38.683699 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 743/1506] 10.222.0.52 () {52 vars in 830 bytes} [Mon Aug 29 22:44:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:44:38.684959 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6325660-45e6-4de4-959b-208cb549000a service cinder] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:38.686263 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-a6325660-45e6-4de4-959b-208cb549000a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:38.786733 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6325660-45e6-4de4-959b-208cb549000a service cinder] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:38.879032 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f00bbf-3a2a-445f-bf3f-5b4d4c5862b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:38.879648 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f00bbf-3a2a-445f-bf3f-5b4d4c5862b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:38.879926 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f00bbf-3a2a-445f-bf3f-5b4d4c5862b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:38.880198 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f00bbf-3a2a-445f-bf3f-5b4d4c5862b2 admin admin] PATH_INFO: `/v3/users/636291677b504c17b49faa2b08bb32b1` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:38.892655 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f00bbf-3a2a-445f-bf3f-5b4d4c5862b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=636291677b504c17b49faa2b08bb32b1)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:38.893696 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: WARNING py.warnings [None req-09f00bbf-3a2a-445f-bf3f-5b4d4c5862b2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 22:44:38.893696 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: warnings.warn(msg) Aug 29 22:44:38.893696 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: Aug 29 22:44:38.894898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f00bbf-3a2a-445f-bf3f-5b4d4c5862b2 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:38.904956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:39.099240 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-09f00bbf-3a2a-445f-bf3f-5b4d4c5862b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '636291677b504c17b49faa2b08bb32b1'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:39.174151 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-09f00bbf-3a2a-445f-bf3f-5b4d4c5862b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '636291677b504c17b49faa2b08bb32b1'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:39.198106 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-09f00bbf-3a2a-445f-bf3f-5b4d4c5862b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '636291677b504c17b49faa2b08bb32b1'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:39.212539 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-09f00bbf-3a2a-445f-bf3f-5b4d4c5862b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '636291677b504c17b49faa2b08bb32b1'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:39.213142 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-09f00bbf-3a2a-445f-bf3f-5b4d4c5862b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '636291677b504c17b49faa2b08bb32b1'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:39.218604 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 764/1507] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:44:38 2022] DELETE /identity/v3/users/636291677b504c17b49faa2b08bb32b1 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:39.222969 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188efd7-a606-4b85-b136-58640ab69d71 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:39.223259 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188efd7-a606-4b85-b136-58640ab69d71 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:39.223538 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188efd7-a606-4b85-b136-58640ab69d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:39.281421 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-2188efd7-a606-4b85-b136-58640ab69d71 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:39.282824 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-2188efd7-a606-4b85-b136-58640ab69d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:39.361898 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:39.374736 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 744/1508] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:44:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:39.384945 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f00bbf-3a2a-445f-bf3f-5b4d4c5862b2 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:39.386153 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-09f00bbf-3a2a-445f-bf3f-5b4d4c5862b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:39.487277 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f00bbf-3a2a-445f-bf3f-5b4d4c5862b2 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:39.545599 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:39.565616 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b9aaa2-720f-4301-b4bd-a99dbe87ca3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:39.566251 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b9aaa2-720f-4301-b4bd-a99dbe87ca3c service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:39.566519 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b9aaa2-720f-4301-b4bd-a99dbe87ca3c service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:39.566785 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b9aaa2-720f-4301-b4bd-a99dbe87ca3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:39.568453 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-87b9aaa2-720f-4301-b4bd-a99dbe87ca3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:39.685191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b9aaa2-720f-4301-b4bd-a99dbe87ca3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:39.686592 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b9aaa2-720f-4301-b4bd-a99dbe87ca3c service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:39.705673 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 765/1509] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:44:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:44:40.142814 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2188efd7-a606-4b85-b136-58640ab69d71 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:40.144128 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-2188efd7-a606-4b85-b136-58640ab69d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:40.156030 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2188efd7-a606-4b85-b136-58640ab69d71 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:40.256031 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b212c-a90a-42d8-912d-b34c73e535fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:40.256579 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b212c-a90a-42d8-912d-b34c73e535fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:40.256913 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b212c-a90a-42d8-912d-b34c73e535fd admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:40.257185 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b212c-a90a-42d8-912d-b34c73e535fd admin admin] PATH_INFO: `/v3/projects/2ab7617cd5d54caea5e6b4fd2cabdae6` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:40.267445 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0b212c-a90a-42d8-912d-b34c73e535fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ab7617cd5d54caea5e6b4fd2cabdae6)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:40.268463 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: WARNING py.warnings [None req-1b0b212c-a90a-42d8-912d-b34c73e535fd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 22:44:40.268463 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: warnings.warn(msg) Aug 29 22:44:40.268463 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: Aug 29 22:44:40.269711 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0b212c-a90a-42d8-912d-b34c73e535fd admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:40.361528 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-1b0b212c-a90a-42d8-912d-b34c73e535fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ab7617cd5d54caea5e6b4fd2cabdae6', 'request_id': 'req-1b0b212c-a90a-42d8-912d-b34c73e535fd'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:40.431173 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-1b0b212c-a90a-42d8-912d-b34c73e535fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ab7617cd5d54caea5e6b4fd2cabdae6', 'request_id': 'req-1b0b212c-a90a-42d8-912d-b34c73e535fd'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:40.438572 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-1b0b212c-a90a-42d8-912d-b34c73e535fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ab7617cd5d54caea5e6b4fd2cabdae6', 'request_id': 'req-1b0b212c-a90a-42d8-912d-b34c73e535fd'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:40.468972 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-1b0b212c-a90a-42d8-912d-b34c73e535fd admin admin] The token cache is being invalidate because project 2ab7617cd5d54caea5e6b4fd2cabdae6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:44:40.469261 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-1b0b212c-a90a-42d8-912d-b34c73e535fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:40.469786 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-1b0b212c-a90a-42d8-912d-b34c73e535fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:40.471088 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 745/1510] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:44:40 2022] DELETE /identity/v3/projects/2ab7617cd5d54caea5e6b4fd2cabdae6 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:44:49.703392 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=62 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:50.466935 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:46:12.017744 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3817f6b-1b1d-40ca-a9be-5852fca288f3 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:12.018341 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3817f6b-1b1d-40ca-a9be-5852fca288f3 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:12.018341 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3817f6b-1b1d-40ca-a9be-5852fca288f3 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:12.019674 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 766/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 22:46:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:46:12.025549 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6469a-d936-4b28-b155-1aca5990c7a4 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:12.026032 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6469a-d936-4b28-b155-1aca5990c7a4 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:12.026442 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6469a-d936-4b28-b155-1aca5990c7a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:12.109232 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-efd6469a-d936-4b28-b155-1aca5990c7a4 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:46:12.110451 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-efd6469a-d936-4b28-b155-1aca5990c7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:12.214216 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 746/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 22:46:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:46:12.372549 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3817f6b-1b1d-40ca-a9be-5852fca288f3 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:46:12.374707 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-c3817f6b-1b1d-40ca-a9be-5852fca288f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:12.495202 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3817f6b-1b1d-40ca-a9be-5852fca288f3 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:46:12.500824 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8c4ec4-72ac-43ed-859a-52cb9b07b000 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:46:12.501430 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c4ec4-72ac-43ed-859a-52cb9b07b000 service nova] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:12.501715 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c4ec4-72ac-43ed-859a-52cb9b07b000 service nova] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:12.501992 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c4ec4-72ac-43ed-859a-52cb9b07b000 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:12.510444 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8c4ec4-72ac-43ed-859a-52cb9b07b000 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:46:12.512279 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8c4ec4-72ac-43ed-859a-52cb9b07b000 service nova] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:46:12.522221 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 767/1513] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:46:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:46:13.374697 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862a8e7-3bb0-4c8d-b852-96e8f0987fce None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:13.375214 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862a8e7-3bb0-4c8d-b852-96e8f0987fce None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:13.375214 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862a8e7-3bb0-4c8d-b852-96e8f0987fce None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:13.376522 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 747/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 22:46:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:46:13.381972 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b0ec7-69fc-4c71-830d-4da135a57559 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:13.382376 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b0ec7-69fc-4c71-830d-4da135a57559 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:13.382535 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b0ec7-69fc-4c71-830d-4da135a57559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:13.402653 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-bd0b0ec7-69fc-4c71-830d-4da135a57559 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:46:13.403852 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-bd0b0ec7-69fc-4c71-830d-4da135a57559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:13.412178 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 768/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 22:46:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:46:13.565985 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2862a8e7-3bb0-4c8d-b852-96e8f0987fce None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:46:13.581217 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2862a8e7-3bb0-4c8d-b852-96e8f0987fce None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:46:13.587527 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d866d67-6454-4ca2-8944-cddafad1b870 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:46:13.588123 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d866d67-6454-4ca2-8944-cddafad1b870 service nova] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:13.588391 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d866d67-6454-4ca2-8944-cddafad1b870 service nova] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:13.588661 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d866d67-6454-4ca2-8944-cddafad1b870 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:13.596801 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d866d67-6454-4ca2-8944-cddafad1b870 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:46:13.598464 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d866d67-6454-4ca2-8944-cddafad1b870 service nova] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:46:13.608011 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 748/1516] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:46:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:46:14.253622 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cbad44-1f68-4706-9d21-cef0748c14f1 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:14.253904 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cbad44-1f68-4706-9d21-cef0748c14f1 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:14.254184 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cbad44-1f68-4706-9d21-cef0748c14f1 None None] PATH_INFO: `/v3` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:14.255679 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 769/1517] 10.222.0.71 () {54 vars in 925 bytes} [Mon Aug 29 22:46:14 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:46:14.262184 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23ae95-e739-401d-82e8-c2085c4a9ee4 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:14.262473 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23ae95-e739-401d-82e8-c2085c4a9ee4 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:14.262750 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23ae95-e739-401d-82e8-c2085c4a9ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:14.442766 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-2e23ae95-e739-401d-82e8-c2085c4a9ee4 None None] MFA Rules not processed for user `d8fd0f6e5b144d2ca78b8d96722502a7`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:46:14.444540 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-2e23ae95-e739-401d-82e8-c2085c4a9ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:14.462339 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:46:14.515898 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 749/1518] 10.222.0.71 () {58 vars in 1015 bytes} [Mon Aug 29 22:46:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:46:18.531062 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe7eb0-ef3f-4a75-b41d-9648cd8a934f None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:18.531638 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe7eb0-ef3f-4a75-b41d-9648cd8a934f None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:18.531751 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe7eb0-ef3f-4a75-b41d-9648cd8a934f None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:18.533076 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 770/1519] 10.222.0.52 () {54 vars in 961 bytes} [Mon Aug 29 22:46:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:46:18.538369 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ab2e8-3715-4358-a26a-ce8fd1b82de0 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:18.538820 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ab2e8-3715-4358-a26a-ce8fd1b82de0 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:18.538932 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ab2e8-3715-4358-a26a-ce8fd1b82de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:18.560286 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-ea2ab2e8-3715-4358-a26a-ce8fd1b82de0 None None] MFA Rules not processed for user `d8fd0f6e5b144d2ca78b8d96722502a7`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:46:18.561556 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-ea2ab2e8-3715-4358-a26a-ce8fd1b82de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:18.571785 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 750/1520] 10.222.0.52 () {58 vars in 1058 bytes} [Mon Aug 29 22:46:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:46:18.579600 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fe7eb0-ef3f-4a75-b41d-9648cd8a934f None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:46:18.593130 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fe7eb0-ef3f-4a75-b41d-9648cd8a934f None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:46:18.598229 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55abad73-1de5-4e00-a698-70bc49babf21 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8fd0f6e5b144d2ca78b8d96722502a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:46:18.598824 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55abad73-1de5-4e00-a698-70bc49babf21 service placement] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:18.599101 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55abad73-1de5-4e00-a698-70bc49babf21 service placement] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:18.599371 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55abad73-1de5-4e00-a698-70bc49babf21 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:18.608054 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55abad73-1de5-4e00-a698-70bc49babf21 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:46:18.609894 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55abad73-1de5-4e00-a698-70bc49babf21 service placement] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:46:18.620564 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 771/1521] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:46:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:46:24.843243 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b8a70-27cf-4ec2-8c5b-af88cb668eb1 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:24.843758 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b8a70-27cf-4ec2-8c5b-af88cb668eb1 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:24.843758 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b8a70-27cf-4ec2-8c5b-af88cb668eb1 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:24.845150 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 751/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 22:46:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:46:24.850199 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e5d0b1-8c09-44fc-83b8-5285b2707100 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:24.850651 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e5d0b1-8c09-44fc-83b8-5285b2707100 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:24.851295 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e5d0b1-8c09-44fc-83b8-5285b2707100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:24.871426 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-b6e5d0b1-8c09-44fc-83b8-5285b2707100 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:46:24.872631 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-b6e5d0b1-8c09-44fc-83b8-5285b2707100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:24.880536 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 772/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 22:46:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:46:25.034179 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b8a70-27cf-4ec2-8c5b-af88cb668eb1 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:46:25.049938 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b8a70-27cf-4ec2-8c5b-af88cb668eb1 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:46:25.055459 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af20cfd-0600-43ea-a3e6-447a71edba86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:46:25.056046 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af20cfd-0600-43ea-a3e6-447a71edba86 service nova] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:25.056326 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af20cfd-0600-43ea-a3e6-447a71edba86 service nova] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:25.056640 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af20cfd-0600-43ea-a3e6-447a71edba86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:25.064842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af20cfd-0600-43ea-a3e6-447a71edba86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:46:25.066444 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af20cfd-0600-43ea-a3e6-447a71edba86 service nova] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:46:25.076585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 752/1524] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:46:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:46:25.901328 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891fd5d5-6502-499c-832f-772a4ed2aa77 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:25.901890 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891fd5d5-6502-499c-832f-772a4ed2aa77 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:25.901890 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891fd5d5-6502-499c-832f-772a4ed2aa77 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:25.903137 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 773/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 22:46:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:46:25.907929 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c013c31-958a-4f29-987b-52efb0cd4fcf None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:25.908327 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c013c31-958a-4f29-987b-52efb0cd4fcf None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:25.908480 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c013c31-958a-4f29-987b-52efb0cd4fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:25.927948 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-2c013c31-958a-4f29-987b-52efb0cd4fcf None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:46:25.929703 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-2c013c31-958a-4f29-987b-52efb0cd4fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:25.937890 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 753/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 22:46:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:46:26.099366 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891fd5d5-6502-499c-832f-772a4ed2aa77 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:46:26.114896 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891fd5d5-6502-499c-832f-772a4ed2aa77 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:46:26.121621 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9405eb-d5ac-4bff-9ac5-0d1977c1345e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:46:26.122216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9405eb-d5ac-4bff-9ac5-0d1977c1345e service nova] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:26.122503 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9405eb-d5ac-4bff-9ac5-0d1977c1345e service nova] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:26.122778 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9405eb-d5ac-4bff-9ac5-0d1977c1345e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:26.130808 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9405eb-d5ac-4bff-9ac5-0d1977c1345e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:46:26.132904 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9405eb-d5ac-4bff-9ac5-0d1977c1345e service nova] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:46:26.142205 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 774/1527] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:46:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:46:30.050773 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2044efbf-73a2-418f-8bf9-eab3805c7658 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:30.051317 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2044efbf-73a2-418f-8bf9-eab3805c7658 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:30.051457 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2044efbf-73a2-418f-8bf9-eab3805c7658 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:30.052816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 754/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 22:46:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:46:30.057996 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e699f-2ab7-41b8-8342-cb18db2ae079 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:30.058435 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e699f-2ab7-41b8-8342-cb18db2ae079 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:30.058534 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e699f-2ab7-41b8-8342-cb18db2ae079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:30.078485 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-402e699f-2ab7-41b8-8342-cb18db2ae079 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:46:30.079681 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-402e699f-2ab7-41b8-8342-cb18db2ae079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:30.089531 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 775/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 22:46:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:46:30.375091 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2044efbf-73a2-418f-8bf9-eab3805c7658 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:46:30.376816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-2044efbf-73a2-418f-8bf9-eab3805c7658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:30.457164 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2044efbf-73a2-418f-8bf9-eab3805c7658 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:46:30.462108 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c10cb40-b2e5-4de1-8888-3431558756c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:46:30.469974 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c10cb40-b2e5-4de1-8888-3431558756c5 service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:30.469974 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c10cb40-b2e5-4de1-8888-3431558756c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:30.469974 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c10cb40-b2e5-4de1-8888-3431558756c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:30.472337 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c10cb40-b2e5-4de1-8888-3431558756c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:46:30.473791 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c10cb40-b2e5-4de1-8888-3431558756c5 service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:46:30.483776 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 755/1530] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:46:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:46:32.394989 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dd3cd2-5c82-42da-a2dd-7f37b59eb018 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:32.395551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dd3cd2-5c82-42da-a2dd-7f37b59eb018 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:32.395551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dd3cd2-5c82-42da-a2dd-7f37b59eb018 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:32.396926 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 776/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 22:46:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:46:32.402654 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60d715-1afd-49cc-8828-d29a84255050 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:32.403052 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60d715-1afd-49cc-8828-d29a84255050 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:32.403193 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60d715-1afd-49cc-8828-d29a84255050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:32.422136 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-fb60d715-1afd-49cc-8828-d29a84255050 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:46:32.423310 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-fb60d715-1afd-49cc-8828-d29a84255050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:32.431421 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 756/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 22:46:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:46:32.718848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dd3cd2-5c82-42da-a2dd-7f37b59eb018 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:46:32.729666 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dd3cd2-5c82-42da-a2dd-7f37b59eb018 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:46:32.733213 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053d097d-81ed-40a5-a87f-8a6c121ed3d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:46:32.733807 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d097d-81ed-40a5-a87f-8a6c121ed3d9 service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:32.734089 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d097d-81ed-40a5-a87f-8a6c121ed3d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:32.734365 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d097d-81ed-40a5-a87f-8a6c121ed3d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:32.744153 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053d097d-81ed-40a5-a87f-8a6c121ed3d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:46:32.745619 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053d097d-81ed-40a5-a87f-8a6c121ed3d9 service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:46:32.755217 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 777/1533] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:46:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:46:36.842748 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1932656-566c-45b7-9ec7-bf4e8432f853 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:36.843336 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1932656-566c-45b7-9ec7-bf4e8432f853 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:36.843336 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1932656-566c-45b7-9ec7-bf4e8432f853 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:36.844548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 757/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 22:46:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:46:36.850126 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d139a-ca71-415d-ac49-b87fc1c15309 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:36.850556 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d139a-ca71-415d-ac49-b87fc1c15309 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:36.850689 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d139a-ca71-415d-ac49-b87fc1c15309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:36.869954 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-c29d139a-ca71-415d-ac49-b87fc1c15309 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:46:36.871146 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-c29d139a-ca71-415d-ac49-b87fc1c15309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:36.878934 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 778/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 22:46:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:46:37.450279 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1932656-566c-45b7-9ec7-bf4e8432f853 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:46:37.451983 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-c1932656-566c-45b7-9ec7-bf4e8432f853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:37.517506 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:46:37.530260 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1932656-566c-45b7-9ec7-bf4e8432f853 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:46:37.533834 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83182c24-73da-437f-84c6-120f470415cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:46:37.534431 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83182c24-73da-437f-84c6-120f470415cc service cinder] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:37.534706 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83182c24-73da-437f-84c6-120f470415cc service cinder] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:37.534972 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83182c24-73da-437f-84c6-120f470415cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:37.546617 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83182c24-73da-437f-84c6-120f470415cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:46:37.548054 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83182c24-73da-437f-84c6-120f470415cc service cinder] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:46:37.558492 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 758/1536] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:46:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:46:39.488448 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58b46f-ad4b-4850-84ef-af5b3cbd4dc6 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:39.489126 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58b46f-ad4b-4850-84ef-af5b3cbd4dc6 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:39.489126 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58b46f-ad4b-4850-84ef-af5b3cbd4dc6 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:39.490435 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 779/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 22:46:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:46:39.495892 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f92187-4828-42f9-bbe6-d937df09e0d5 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:39.496359 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f92187-4828-42f9-bbe6-d937df09e0d5 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:39.496466 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f92187-4828-42f9-bbe6-d937df09e0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:39.517707 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-41f92187-4828-42f9-bbe6-d937df09e0d5 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:46:39.518902 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-41f92187-4828-42f9-bbe6-d937df09e0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:39.527394 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 759/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 22:46:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:46:40.078479 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58b46f-ad4b-4850-84ef-af5b3cbd4dc6 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:46:40.080205 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-0c58b46f-ad4b-4850-84ef-af5b3cbd4dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:40.087965 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=84 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:46:40.093768 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58b46f-ad4b-4850-84ef-af5b3cbd4dc6 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:46:40.097386 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76c4dfd-a9df-4b5a-abd6-76e6ac9d02c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:46:40.104312 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c4dfd-a9df-4b5a-abd6-76e6ac9d02c8 service cinder] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:40.104312 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c4dfd-a9df-4b5a-abd6-76e6ac9d02c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:40.104312 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c4dfd-a9df-4b5a-abd6-76e6ac9d02c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:40.107673 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76c4dfd-a9df-4b5a-abd6-76e6ac9d02c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:46:40.109153 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76c4dfd-a9df-4b5a-abd6-76e6ac9d02c8 service cinder] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:46:40.119574 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 780/1539] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:46:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:46:49.512301 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=11 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:46:50.127394 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=5 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:10.982238 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f43df34-63bf-4cc4-965f-840c24fae28c None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:10.984008 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f43df34-63bf-4cc4-965f-840c24fae28c None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:10.984521 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f43df34-63bf-4cc4-965f-840c24fae28c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:11.011069 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-8f43df34-63bf-4cc4-965f-840c24fae28c None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:11.012645 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-8f43df34-63bf-4cc4-965f-840c24fae28c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:11.024654 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 760/1540] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:11.034360 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6feb034-16e2-4acf-a497-8af221fa89d7 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:11.034970 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6feb034-16e2-4acf-a497-8af221fa89d7 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:11.034970 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6feb034-16e2-4acf-a497-8af221fa89d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:11.063614 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-e6feb034-16e2-4acf-a497-8af221fa89d7 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:11.064958 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-e6feb034-16e2-4acf-a497-8af221fa89d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:11.083906 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 781/1541] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:11.088711 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f43df34-63bf-4cc4-965f-840c24fae28c None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:11.101947 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f43df34-63bf-4cc4-965f-840c24fae28c None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:11.109662 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aed189-22c2-4b4b-a80d-07d673ca804f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:11.110369 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aed189-22c2-4b4b-a80d-07d673ca804f admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:11.110663 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aed189-22c2-4b4b-a80d-07d673ca804f admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:11.110941 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aed189-22c2-4b4b-a80d-07d673ca804f admin admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:11.111628 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aed189-22c2-4b4b-a80d-07d673ca804f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:48:11.111936 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aed189-22c2-4b4b-a80d-07d673ca804f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:11.113621 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aed189-22c2-4b4b-a80d-07d673ca804f admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:11.130712 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 761/1542] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:48:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:48:11.139021 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6feb034-16e2-4acf-a497-8af221fa89d7 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:11.149907 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6feb034-16e2-4acf-a497-8af221fa89d7 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:11.153090 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49608d7e-61c3-41a3-9845-bfec82e1e3d6 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:11.154553 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49608d7e-61c3-41a3-9845-bfec82e1e3d6 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:11.154553 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49608d7e-61c3-41a3-9845-bfec82e1e3d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:11.163901 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a26310-a34b-4108-b8b9-c32530f048f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:11.165947 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a26310-a34b-4108-b8b9-c32530f048f2 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:11.166454 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a26310-a34b-4108-b8b9-c32530f048f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:11.168310 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a26310-a34b-4108-b8b9-c32530f048f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:11.169209 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a26310-a34b-4108-b8b9-c32530f048f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:11.171402 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a26310-a34b-4108-b8b9-c32530f048f2 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:11.196655 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-49608d7e-61c3-41a3-9845-bfec82e1e3d6 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:11.200973 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-49608d7e-61c3-41a3-9845-bfec82e1e3d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:11.228851 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 762/1543] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 29 22:48:11.230131 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfabe60-e134-4afe-b2fa-07313a0a5a8f None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:11.230421 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfabe60-e134-4afe-b2fa-07313a0a5a8f None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:11.230697 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfabe60-e134-4afe-b2fa-07313a0a5a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:11.260256 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-cbfabe60-e134-4afe-b2fa-07313a0a5a8f None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:11.260256 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-cbfabe60-e134-4afe-b2fa-07313a0a5a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:11.268003 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 763/1544] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:11.269331 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d9ee9-68eb-4afc-9273-775cf7818ec2 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:11.269618 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d9ee9-68eb-4afc-9273-775cf7818ec2 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:11.269942 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d9ee9-68eb-4afc-9273-775cf7818ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:11.301699 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-fa7d9ee9-68eb-4afc-9273-775cf7818ec2 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:11.302935 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-fa7d9ee9-68eb-4afc-9273-775cf7818ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:11.314138 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 764/1545] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:11.320496 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d148c-d562-418a-8785-f94d59b5e588 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:11.320496 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d148c-d562-418a-8785-f94d59b5e588 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:11.320496 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d148c-d562-418a-8785-f94d59b5e588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:11.320809 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 782/1546] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:48:11 2022] POST /identity/v3/projects => generated 381 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:48:11.322004 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e6595e-a3b7-4fd3-b079-b36dfc11a507 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:11.323007 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e6595e-a3b7-4fd3-b079-b36dfc11a507 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:11.323773 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e6595e-a3b7-4fd3-b079-b36dfc11a507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:11.349938 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-278d148c-d562-418a-8785-f94d59b5e588 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:11.351276 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-278d148c-d562-418a-8785-f94d59b5e588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:11.354764 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-71e6595e-a3b7-4fd3-b079-b36dfc11a507 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:11.355939 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-71e6595e-a3b7-4fd3-b079-b36dfc11a507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:11.429318 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 765/1547] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:11.430408 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1432f0-cc3f-484e-b18c-7dfc17e94a83 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:11.430694 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1432f0-cc3f-484e-b18c-7dfc17e94a83 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:11.430960 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1432f0-cc3f-484e-b18c-7dfc17e94a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:11.435710 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 783/1548] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:11.435710 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd5ff7-0bf0-4470-a97a-4f1cb74b465c None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:11.435710 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd5ff7-0bf0-4470-a97a-4f1cb74b465c None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:11.435710 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd5ff7-0bf0-4470-a97a-4f1cb74b465c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:11.454835 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-a2fd5ff7-0bf0-4470-a97a-4f1cb74b465c None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:11.456038 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-a2fd5ff7-0bf0-4470-a97a-4f1cb74b465c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:11.458880 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-5d1432f0-cc3f-484e-b18c-7dfc17e94a83 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:11.460049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-5d1432f0-cc3f-484e-b18c-7dfc17e94a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:11.472665 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 784/1549] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:11.473477 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fd5ff7-0bf0-4470-a97a-4f1cb74b465c None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:11.475324 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 766/1550] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:11.475324 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efadf00-c97a-45eb-b7c6-11b988b9c9da None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:11.475598 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efadf00-c97a-45eb-b7c6-11b988b9c9da None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:11.478357 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efadf00-c97a-45eb-b7c6-11b988b9c9da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:11.483411 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fd5ff7-0bf0-4470-a97a-4f1cb74b465c None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:11.490731 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe1e49-1d74-4ed5-8193-dc11901f64a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:11.491343 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe1e49-1d74-4ed5-8193-dc11901f64a0 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:11.491622 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe1e49-1d74-4ed5-8193-dc11901f64a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:11.491893 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe1e49-1d74-4ed5-8193-dc11901f64a0 admin admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:11.492518 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fe1e49-1d74-4ed5-8193-dc11901f64a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:11.493563 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: WARNING py.warnings [None req-50fe1e49-1d74-4ed5-8193-dc11901f64a0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 22:48:11.493563 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: warnings.warn(msg) Aug 29 22:48:11.493563 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: Aug 29 22:48:11.494775 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fe1e49-1d74-4ed5-8193-dc11901f64a0 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:11.496118 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-4efadf00-c97a-45eb-b7c6-11b988b9c9da None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:11.497356 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-4efadf00-c97a-45eb-b7c6-11b988b9c9da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:11.506204 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 767/1551] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:11.507428 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39ab7f-a628-4609-8e49-a19e1fa65cd3 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:11.507710 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39ab7f-a628-4609-8e49-a19e1fa65cd3 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:11.507981 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39ab7f-a628-4609-8e49-a19e1fa65cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:11.527441 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-ba39ab7f-a628-4609-8e49-a19e1fa65cd3 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:11.528542 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-ba39ab7f-a628-4609-8e49-a19e1fa65cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:11.538063 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 768/1552] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:11.539277 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a41357-1f70-4ff5-8172-8d4b5aca08f9 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:11.539562 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a41357-1f70-4ff5-8172-8d4b5aca08f9 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:11.539837 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a41357-1f70-4ff5-8172-8d4b5aca08f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:11.560288 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-a0a41357-1f70-4ff5-8172-8d4b5aca08f9 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:11.561702 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-a0a41357-1f70-4ff5-8172-8d4b5aca08f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:11.569898 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 769/1553] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:11.571528 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a41357-1f70-4ff5-8172-8d4b5aca08f9 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:11.581878 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a41357-1f70-4ff5-8172-8d4b5aca08f9 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:11.586584 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca986c12-c904-4793-81c7-dfd3a24a11ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:11.587272 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca986c12-c904-4793-81c7-dfd3a24a11ee admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:11.587556 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca986c12-c904-4793-81c7-dfd3a24a11ee admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:11.587827 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca986c12-c904-4793-81c7-dfd3a24a11ee admin admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:11.588502 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca986c12-c904-4793-81c7-dfd3a24a11ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:48:11.588836 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca986c12-c904-4793-81c7-dfd3a24a11ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:11.590429 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca986c12-c904-4793-81c7-dfd3a24a11ee admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:11.602924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 785/1554] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:48:11 2022] POST /identity/v3/users => generated 291 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:48:11.610565 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 770/1555] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:48:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:48:11.610868 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f8217-7835-4df4-898b-2ec281c6c463 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:11.610868 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f8217-7835-4df4-898b-2ec281c6c463 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:11.610868 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f8217-7835-4df4-898b-2ec281c6c463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:11.611271 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca986c12-c904-4793-81c7-dfd3a24a11ee admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:11.619847 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca986c12-c904-4793-81c7-dfd3a24a11ee admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:11.624749 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a95dbb-b527-4d7a-b089-42cb18346580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:11.625430 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a95dbb-b527-4d7a-b089-42cb18346580 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:11.625725 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a95dbb-b527-4d7a-b089-42cb18346580 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:11.626005 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a95dbb-b527-4d7a-b089-42cb18346580 admin admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:11.626266 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-543f8217-7835-4df4-898b-2ec281c6c463 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:11.626690 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a95dbb-b527-4d7a-b089-42cb18346580 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:48:11.626993 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a95dbb-b527-4d7a-b089-42cb18346580 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:11.627496 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-543f8217-7835-4df4-898b-2ec281c6c463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:11.628580 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a95dbb-b527-4d7a-b089-42cb18346580 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:11.639339 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 786/1556] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:11.639339 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b616942-ded4-4019-9035-81f4eada6945 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:11.639339 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b616942-ded4-4019-9035-81f4eada6945 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:11.639339 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b616942-ded4-4019-9035-81f4eada6945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:11.645811 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 771/1557] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:48:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:48:11.647142 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a95dbb-b527-4d7a-b089-42cb18346580 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:11.658440 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-5b616942-ded4-4019-9035-81f4eada6945 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:11.659346 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a95dbb-b527-4d7a-b089-42cb18346580 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:11.659642 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-5b616942-ded4-4019-9035-81f4eada6945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:11.663949 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e21bed-6208-4cf1-9ddb-b54a8a737509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:11.664572 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e21bed-6208-4cf1-9ddb-b54a8a737509 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:11.664896 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e21bed-6208-4cf1-9ddb-b54a8a737509 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:11.665167 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e21bed-6208-4cf1-9ddb-b54a8a737509 admin admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:11.665833 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e21bed-6208-4cf1-9ddb-b54a8a737509 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:48:11.666130 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e21bed-6208-4cf1-9ddb-b54a8a737509 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:11.667637 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 787/1558] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:11.667773 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e21bed-6208-4cf1-9ddb-b54a8a737509 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:11.668867 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd329c-8ab7-4822-bdea-c56f94ea9f8d None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:11.669154 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd329c-8ab7-4822-bdea-c56f94ea9f8d None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:11.669432 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd329c-8ab7-4822-bdea-c56f94ea9f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:11.690219 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 772/1559] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:48:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:48:11.690219 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ba5bdb-502b-4705-ac76-b2ba0e072f82 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:11.690219 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ba5bdb-502b-4705-ac76-b2ba0e072f82 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:11.690219 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ba5bdb-502b-4705-ac76-b2ba0e072f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:11.690698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-f9cd329c-8ab7-4822-bdea-c56f94ea9f8d None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:11.690698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-f9cd329c-8ab7-4822-bdea-c56f94ea9f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:11.698599 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 788/1560] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:11.700551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd329c-8ab7-4822-bdea-c56f94ea9f8d None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:11.706331 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-c0ba5bdb-502b-4705-ac76-b2ba0e072f82 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:11.707793 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-c0ba5bdb-502b-4705-ac76-b2ba0e072f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:11.711133 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd329c-8ab7-4822-bdea-c56f94ea9f8d None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:11.715817 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 773/1561] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:11.716020 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f010696-47c5-409a-8924-1048ae594675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:11.716619 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f010696-47c5-409a-8924-1048ae594675 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:11.716906 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ba5bdb-502b-4705-ac76-b2ba0e072f82 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:11.717005 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f010696-47c5-409a-8924-1048ae594675 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:11.717280 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f010696-47c5-409a-8924-1048ae594675 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:11.717966 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f010696-47c5-409a-8924-1048ae594675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:11.720387 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f010696-47c5-409a-8924-1048ae594675 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:11.727159 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ba5bdb-502b-4705-ac76-b2ba0e072f82 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:11.730952 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 789/1562] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:48:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:48:11.731621 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f010696-47c5-409a-8924-1048ae594675 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:11.734540 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3be032-f901-4a78-b538-c51af1af927d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:11.735243 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3be032-f901-4a78-b538-c51af1af927d admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:11.735526 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3be032-f901-4a78-b538-c51af1af927d admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:11.735802 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3be032-f901-4a78-b538-c51af1af927d admin admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:11.736441 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3be032-f901-4a78-b538-c51af1af927d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:11.737493 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: WARNING py.warnings [None req-7b3be032-f901-4a78-b538-c51af1af927d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 22:48:11.737493 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: warnings.warn(msg) Aug 29 22:48:11.737493 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: Aug 29 22:48:11.738741 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3be032-f901-4a78-b538-c51af1af927d admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:11.741575 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f010696-47c5-409a-8924-1048ae594675 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:11.746269 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b105539-8943-444d-b9e9-32c9625c8b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:11.746856 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b105539-8943-444d-b9e9-32c9625c8b39 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:11.747143 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b105539-8943-444d-b9e9-32c9625c8b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:11.747419 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b105539-8943-444d-b9e9-32c9625c8b39 admin admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:11.748087 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b105539-8943-444d-b9e9-32c9625c8b39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:48:11.748393 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b105539-8943-444d-b9e9-32c9625c8b39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:11.749760 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: WARNING py.warnings [None req-7b105539-8943-444d-b9e9-32c9625c8b39 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 29 22:48:11.749760 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: warnings.warn(msg) Aug 29 22:48:11.749760 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: Aug 29 22:48:11.750672 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b105539-8943-444d-b9e9-32c9625c8b39 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:11.762795 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 790/1563] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:48:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:48:11.763604 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b105539-8943-444d-b9e9-32c9625c8b39 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:11.774090 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b105539-8943-444d-b9e9-32c9625c8b39 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:11.778805 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea2c46-0e76-4105-818a-a9f6183a1314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:11.779326 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea2c46-0e76-4105-818a-a9f6183a1314 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:11.779630 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea2c46-0e76-4105-818a-a9f6183a1314 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:11.779887 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea2c46-0e76-4105-818a-a9f6183a1314 admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:11.781447 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aea2c46-0e76-4105-818a-a9f6183a1314 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:11.783173 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aea2c46-0e76-4105-818a-a9f6183a1314 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:11.851647 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 774/1564] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:48:11 2022] POST /identity/v3/projects => generated 377 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:48:11.852640 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3be032-f901-4a78-b538-c51af1af927d admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:11.887712 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:11.916879 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 791/1565] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:48:11 2022] POST /identity/v3/projects => generated 365 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:48:11.916879 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea2c46-0e76-4105-818a-a9f6183a1314 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:11.923205 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3be032-f901-4a78-b538-c51af1af927d admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:11.961295 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:11.982293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134386d1-8d15-4fa4-861e-bd989351b66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:11.982908 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134386d1-8d15-4fa4-861e-bd989351b66a admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:11.983210 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134386d1-8d15-4fa4-861e-bd989351b66a admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:11.983535 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134386d1-8d15-4fa4-861e-bd989351b66a admin admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:11.984402 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134386d1-8d15-4fa4-861e-bd989351b66a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:48:11.985004 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134386d1-8d15-4fa4-861e-bd989351b66a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:11.986955 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134386d1-8d15-4fa4-861e-bd989351b66a admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:11.990083 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea2c46-0e76-4105-818a-a9f6183a1314 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:11.994851 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f687faa-e69f-4fd7-8258-25b33326922d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:11.995576 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f687faa-e69f-4fd7-8258-25b33326922d admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:11.995855 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f687faa-e69f-4fd7-8258-25b33326922d admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:11.996131 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f687faa-e69f-4fd7-8258-25b33326922d admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:11.996784 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f687faa-e69f-4fd7-8258-25b33326922d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:11.998763 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f687faa-e69f-4fd7-8258-25b33326922d admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:11.999626 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 775/1566] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:48:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:48:12.001045 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134386d1-8d15-4fa4-861e-bd989351b66a admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:12.011363 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134386d1-8d15-4fa4-861e-bd989351b66a admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:12.017632 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e530269c-7675-4725-aa71-5a4e34fd0964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:12.018079 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e530269c-7675-4725-aa71-5a4e34fd0964 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:12.018436 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e530269c-7675-4725-aa71-5a4e34fd0964 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:12.018664 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e530269c-7675-4725-aa71-5a4e34fd0964 admin admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:12.019311 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e530269c-7675-4725-aa71-5a4e34fd0964 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:48:12.019614 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e530269c-7675-4725-aa71-5a4e34fd0964 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:12.021299 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e530269c-7675-4725-aa71-5a4e34fd0964 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:12.034871 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 776/1567] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:48:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:48:12.035918 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e530269c-7675-4725-aa71-5a4e34fd0964 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:12.045318 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e530269c-7675-4725-aa71-5a4e34fd0964 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:12.049830 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0724f3d6-8bfa-4913-be2e-93f1848cbda2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:12.050396 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0724f3d6-8bfa-4913-be2e-93f1848cbda2 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:12.050668 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0724f3d6-8bfa-4913-be2e-93f1848cbda2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:12.050935 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0724f3d6-8bfa-4913-be2e-93f1848cbda2 admin admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:12.051562 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0724f3d6-8bfa-4913-be2e-93f1848cbda2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:48:12.051855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0724f3d6-8bfa-4913-be2e-93f1848cbda2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:12.053538 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0724f3d6-8bfa-4913-be2e-93f1848cbda2 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:12.065655 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 777/1568] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:48:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:48:12.066473 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0724f3d6-8bfa-4913-be2e-93f1848cbda2 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:12.075790 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0724f3d6-8bfa-4913-be2e-93f1848cbda2 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:12.082174 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8fe66a-3be7-4587-a097-45d29dff6982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:12.082777 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8fe66a-3be7-4587-a097-45d29dff6982 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:12.083056 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8fe66a-3be7-4587-a097-45d29dff6982 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:12.083328 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8fe66a-3be7-4587-a097-45d29dff6982 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:12.083963 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8fe66a-3be7-4587-a097-45d29dff6982 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:12.086139 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 792/1569] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:48:11 2022] POST /identity/v3/projects => generated 373 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:48:12.086649 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8fe66a-3be7-4587-a097-45d29dff6982 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:12.098582 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f687faa-e69f-4fd7-8258-25b33326922d admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:12.098741 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 778/1570] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:48:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:48:12.098741 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8fe66a-3be7-4587-a097-45d29dff6982 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:12.107112 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8fe66a-3be7-4587-a097-45d29dff6982 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:12.152461 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f687faa-e69f-4fd7-8258-25b33326922d admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:12.158101 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6daebc-2278-4a24-85c3-3cd32174967e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:12.158715 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6daebc-2278-4a24-85c3-3cd32174967e admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:12.158998 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6daebc-2278-4a24-85c3-3cd32174967e admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:12.159272 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6daebc-2278-4a24-85c3-3cd32174967e admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:12.159914 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6daebc-2278-4a24-85c3-3cd32174967e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:12.162275 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6daebc-2278-4a24-85c3-3cd32174967e admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:12.167620 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932a0d27-f632-4675-ab00-69b2c239293b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:12.168302 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a0d27-f632-4675-ab00-69b2c239293b admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:12.168632 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a0d27-f632-4675-ab00-69b2c239293b admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:12.168982 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a0d27-f632-4675-ab00-69b2c239293b admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:12.169770 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932a0d27-f632-4675-ab00-69b2c239293b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:12.173495 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932a0d27-f632-4675-ab00-69b2c239293b admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:12.261001 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 779/1571] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:48:12 2022] POST /identity/v3/users => generated 289 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:48:12.261828 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932a0d27-f632-4675-ab00-69b2c239293b admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:12.272008 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932a0d27-f632-4675-ab00-69b2c239293b admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:12.272613 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 793/1572] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:48:12 2022] POST /identity/v3/projects => generated 375 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:48:12.273393 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6daebc-2278-4a24-85c3-3cd32174967e admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:12.293229 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:12.336337 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb127e75-73e2-4645-a8c9-b8b8521b6718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:12.337005 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb127e75-73e2-4645-a8c9-b8b8521b6718 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:12.337305 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb127e75-73e2-4645-a8c9-b8b8521b6718 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:12.337871 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb127e75-73e2-4645-a8c9-b8b8521b6718 admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:12.338818 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb127e75-73e2-4645-a8c9-b8b8521b6718 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:12.340484 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6daebc-2278-4a24-85c3-3cd32174967e admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:12.341282 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb127e75-73e2-4645-a8c9-b8b8521b6718 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:12.345199 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b502aa3-c86f-46be-9839-a1e17dda851d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:12.345856 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b502aa3-c86f-46be-9839-a1e17dda851d admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:12.346141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b502aa3-c86f-46be-9839-a1e17dda851d admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:12.346405 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b502aa3-c86f-46be-9839-a1e17dda851d admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:12.347073 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b502aa3-c86f-46be-9839-a1e17dda851d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:12.348985 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b502aa3-c86f-46be-9839-a1e17dda851d admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:12.444218 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 780/1573] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:48:12 2022] POST /identity/v3/users => generated 283 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:48:12.445211 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb127e75-73e2-4645-a8c9-b8b8521b6718 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:12.455194 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 794/1574] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:48:12 2022] POST /identity/v3/projects => generated 383 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:48:12.455390 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb127e75-73e2-4645-a8c9-b8b8521b6718 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:12.455902 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b502aa3-c86f-46be-9839-a1e17dda851d admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:12.490461 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:12.516095 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9237c449-8e9a-4fbc-9353-74e42f85527a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:12.516902 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9237c449-8e9a-4fbc-9353-74e42f85527a admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:12.517184 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9237c449-8e9a-4fbc-9353-74e42f85527a admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:12.517451 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9237c449-8e9a-4fbc-9353-74e42f85527a admin admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:12.518091 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9237c449-8e9a-4fbc-9353-74e42f85527a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:12.520008 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9237c449-8e9a-4fbc-9353-74e42f85527a admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:12.526641 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b502aa3-c86f-46be-9839-a1e17dda851d admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:12.533086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210d2d0a-a36f-4ed7-873e-716f50319113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:12.533722 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d2d0a-a36f-4ed7-873e-716f50319113 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:12.534004 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d2d0a-a36f-4ed7-873e-716f50319113 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:12.534283 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d2d0a-a36f-4ed7-873e-716f50319113 admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:12.534926 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210d2d0a-a36f-4ed7-873e-716f50319113 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:12.536927 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210d2d0a-a36f-4ed7-873e-716f50319113 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:12.616572 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 781/1575] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:48:12 2022] POST /identity/v3/projects => generated 387 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:48:12.617349 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9237c449-8e9a-4fbc-9353-74e42f85527a admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:12.638631 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 795/1576] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:48:12 2022] POST /identity/v3/projects => generated 393 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:48:12.640394 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210d2d0a-a36f-4ed7-873e-716f50319113 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:12.683626 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9237c449-8e9a-4fbc-9353-74e42f85527a admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:12.706345 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210d2d0a-a36f-4ed7-873e-716f50319113 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:12.711605 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8504115-5d09-4dbd-a48e-f1308794007c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:12.712404 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8504115-5d09-4dbd-a48e-f1308794007c admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:12.712807 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8504115-5d09-4dbd-a48e-f1308794007c admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:12.713091 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8504115-5d09-4dbd-a48e-f1308794007c admin admin] PATH_INFO: `/v3/projects/d223b229017e4f0fb5bea686e62ad992/users/75a78417cb914a10ab735d13b49aecff/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:12.724309 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:12.728608 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8504115-5d09-4dbd-a48e-f1308794007c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d223b229017e4f0fb5bea686e62ad992, user_id=75a78417cb914a10ab735d13b49aecff, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:12.731393 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8504115-5d09-4dbd-a48e-f1308794007c admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:12.740829 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a9731-b3c8-4587-9f5b-9d1ba0b3826e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:12.741608 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a9731-b3c8-4587-9f5b-9d1ba0b3826e admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:12.741731 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a9731-b3c8-4587-9f5b-9d1ba0b3826e admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:12.741975 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a9731-b3c8-4587-9f5b-9d1ba0b3826e admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:12.742601 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903a9731-b3c8-4587-9f5b-9d1ba0b3826e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:12.744574 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903a9731-b3c8-4587-9f5b-9d1ba0b3826e admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:12.809052 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 796/1577] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:48:12 2022] PUT /identity/v3/projects/d223b229017e4f0fb5bea686e62ad992/users/75a78417cb914a10ab735d13b49aecff/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:48:12.810084 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8504115-5d09-4dbd-a48e-f1308794007c admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:12.828580 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 782/1578] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:48:12 2022] POST /identity/v3/users => generated 287 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:48:12.829719 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a9731-b3c8-4587-9f5b-9d1ba0b3826e admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:12.840220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a9731-b3c8-4587-9f5b-9d1ba0b3826e admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:12.880047 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8504115-5d09-4dbd-a48e-f1308794007c admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:12.884642 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b3385a-ca05-4333-abe9-c1b559ac1d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:12.885300 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3385a-ca05-4333-abe9-c1b559ac1d37 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:12.885578 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3385a-ca05-4333-abe9-c1b559ac1d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:12.885865 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3385a-ca05-4333-abe9-c1b559ac1d37 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:12.886496 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b3385a-ca05-4333-abe9-c1b559ac1d37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:12.888100 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b3385a-ca05-4333-abe9-c1b559ac1d37 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:12.899676 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 797/1579] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:48:12 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:48:12.900496 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b3385a-ca05-4333-abe9-c1b559ac1d37 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:12.902666 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256d0033-1a22-4fac-8910-b7e7b9cf02e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:12.903256 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d0033-1a22-4fac-8910-b7e7b9cf02e5 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:12.903539 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d0033-1a22-4fac-8910-b7e7b9cf02e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:12.903815 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d0033-1a22-4fac-8910-b7e7b9cf02e5 admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:12.904447 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256d0033-1a22-4fac-8910-b7e7b9cf02e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:12.907326 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256d0033-1a22-4fac-8910-b7e7b9cf02e5 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:12.910564 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b3385a-ca05-4333-abe9-c1b559ac1d37 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:12.915080 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d873e8-9e5b-403e-abb5-d86baf4b2e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:12.915666 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d873e8-9e5b-403e-abb5-d86baf4b2e5c admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:12.915949 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d873e8-9e5b-403e-abb5-d86baf4b2e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:12.916222 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d873e8-9e5b-403e-abb5-d86baf4b2e5c admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:12.916889 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d873e8-9e5b-403e-abb5-d86baf4b2e5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:12.918464 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d873e8-9e5b-403e-abb5-d86baf4b2e5c admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:12.929100 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 798/1580] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:48:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:48:12.929870 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d873e8-9e5b-403e-abb5-d86baf4b2e5c admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:12.939550 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d873e8-9e5b-403e-abb5-d86baf4b2e5c admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:12.944833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11073b3-2495-4138-8240-1be2997e43cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:12.944833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11073b3-2495-4138-8240-1be2997e43cd admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:12.945150 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11073b3-2495-4138-8240-1be2997e43cd admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:12.945295 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11073b3-2495-4138-8240-1be2997e43cd admin admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:12.945828 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11073b3-2495-4138-8240-1be2997e43cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:12.947692 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11073b3-2495-4138-8240-1be2997e43cd admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:12.986407 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 783/1581] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:48:12 2022] POST /identity/v3/users => generated 288 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:48:12.987522 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256d0033-1a22-4fac-8910-b7e7b9cf02e5 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:12.997149 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256d0033-1a22-4fac-8910-b7e7b9cf02e5 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:13.001895 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb2c40-31b7-4ecb-9c43-a130632e3c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:13.002471 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb2c40-31b7-4ecb-9c43-a130632e3c09 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:13.002745 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb2c40-31b7-4ecb-9c43-a130632e3c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:13.003016 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb2c40-31b7-4ecb-9c43-a130632e3c09 admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:13.003617 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cb2c40-31b7-4ecb-9c43-a130632e3c09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:13.005628 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cb2c40-31b7-4ecb-9c43-a130632e3c09 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:13.031862 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 799/1582] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:48:12 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:48:13.032812 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11073b3-2495-4138-8240-1be2997e43cd admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:13.038804 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:13.042470 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11073b3-2495-4138-8240-1be2997e43cd admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:13.047737 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff9d998-df2f-402d-9aa6-01bcc7252425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:13.048315 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9d998-df2f-402d-9aa6-01bcc7252425 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:13.048597 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9d998-df2f-402d-9aa6-01bcc7252425 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:13.048911 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9d998-df2f-402d-9aa6-01bcc7252425 admin admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:13.049530 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff9d998-df2f-402d-9aa6-01bcc7252425 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:13.051518 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff9d998-df2f-402d-9aa6-01bcc7252425 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:13.078162 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 784/1583] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:48:12 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:48:13.079712 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bba072e-d986-4b9a-afea-e4cff7d60b9e None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:13.079997 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bba072e-d986-4b9a-afea-e4cff7d60b9e None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:13.080282 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bba072e-d986-4b9a-afea-e4cff7d60b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:13.110760 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-8bba072e-d986-4b9a-afea-e4cff7d60b9e None None] MFA Rules not processed for user `75a78417cb914a10ab735d13b49aecff`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:13.111967 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-8bba072e-d986-4b9a-afea-e4cff7d60b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:13.155909 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 800/1584] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:48:13 2022] POST /identity/v3/users => generated 297 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:48:13.155909 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff9d998-df2f-402d-9aa6-01bcc7252425 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:13.157388 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff9d998-df2f-402d-9aa6-01bcc7252425 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:13.161468 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eac57f-8150-4d2d-84b5-8c5ca1e14056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:13.162074 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eac57f-8150-4d2d-84b5-8c5ca1e14056 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:13.162350 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eac57f-8150-4d2d-84b5-8c5ca1e14056 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:13.162622 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eac57f-8150-4d2d-84b5-8c5ca1e14056 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:13.163268 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eac57f-8150-4d2d-84b5-8c5ca1e14056 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:13.164920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eac57f-8150-4d2d-84b5-8c5ca1e14056 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:13.175865 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 801/1585] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:48:13.176688 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eac57f-8150-4d2d-84b5-8c5ca1e14056 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:13.182879 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 785/1586] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:13 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:13.182879 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bba072e-d986-4b9a-afea-e4cff7d60b9e None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:13.186535 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eac57f-8150-4d2d-84b5-8c5ca1e14056 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:13.191214 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec135843-136b-4a39-98ed-314a243b164c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:13.196095 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec135843-136b-4a39-98ed-314a243b164c admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:13.196095 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec135843-136b-4a39-98ed-314a243b164c admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:13.196095 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec135843-136b-4a39-98ed-314a243b164c admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:13.196473 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bba072e-d986-4b9a-afea-e4cff7d60b9e None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:13.196582 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec135843-136b-4a39-98ed-314a243b164c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:13.196582 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec135843-136b-4a39-98ed-314a243b164c admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:13.197470 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d45037-9496-4707-9a95-725192da9476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:13.198061 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d45037-9496-4707-9a95-725192da9476 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:13.198341 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d45037-9496-4707-9a95-725192da9476 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:13.198614 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d45037-9496-4707-9a95-725192da9476 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:13.199255 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d45037-9496-4707-9a95-725192da9476 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:13.200940 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d45037-9496-4707-9a95-725192da9476 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:13.205679 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 802/1587] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:48:13.206488 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec135843-136b-4a39-98ed-314a243b164c admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:13.215108 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 786/1588] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:48:13.215108 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d45037-9496-4707-9a95-725192da9476 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:13.219998 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec135843-136b-4a39-98ed-314a243b164c admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:13.223723 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d45037-9496-4707-9a95-725192da9476 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:13.224756 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1c94c5-6a6c-445e-abae-41149b4f9cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:13.225356 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c94c5-6a6c-445e-abae-41149b4f9cfb admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:13.225637 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c94c5-6a6c-445e-abae-41149b4f9cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:13.225920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c94c5-6a6c-445e-abae-41149b4f9cfb admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:13.226550 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1c94c5-6a6c-445e-abae-41149b4f9cfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:13.228167 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1c94c5-6a6c-445e-abae-41149b4f9cfb admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:13.228523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88edc69-8d66-4ef9-b22e-f50b77e5a988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:13.229195 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88edc69-8d66-4ef9-b22e-f50b77e5a988 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:13.229501 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88edc69-8d66-4ef9-b22e-f50b77e5a988 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:13.229741 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88edc69-8d66-4ef9-b22e-f50b77e5a988 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:13.230381 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88edc69-8d66-4ef9-b22e-f50b77e5a988 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:13.231964 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88edc69-8d66-4ef9-b22e-f50b77e5a988 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:13.239413 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 803/1589] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:48:13.240372 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1c94c5-6a6c-445e-abae-41149b4f9cfb admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:13.244104 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 787/1590] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:48:13.245286 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88edc69-8d66-4ef9-b22e-f50b77e5a988 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:13.253423 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1c94c5-6a6c-445e-abae-41149b4f9cfb admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:13.254841 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88edc69-8d66-4ef9-b22e-f50b77e5a988 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:13.257275 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9e8948-1366-45ea-b627-3343f2e09c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:13.257909 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e8948-1366-45ea-b627-3343f2e09c77 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:13.258186 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e8948-1366-45ea-b627-3343f2e09c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:13.258457 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e8948-1366-45ea-b627-3343f2e09c77 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:13.259094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9e8948-1366-45ea-b627-3343f2e09c77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:13.259523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac2d28-ec9c-4030-8af0-8d6f698e678a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:13.260100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac2d28-ec9c-4030-8af0-8d6f698e678a admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:13.260383 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac2d28-ec9c-4030-8af0-8d6f698e678a admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:13.260657 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac2d28-ec9c-4030-8af0-8d6f698e678a admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:13.260824 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9e8948-1366-45ea-b627-3343f2e09c77 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:13.261322 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bac2d28-ec9c-4030-8af0-8d6f698e678a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:13.262922 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bac2d28-ec9c-4030-8af0-8d6f698e678a admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:13.269331 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:13.271574 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 804/1591] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:48:13.272529 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9e8948-1366-45ea-b627-3343f2e09c77 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:13.279219 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 788/1592] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:48:13.279219 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac2d28-ec9c-4030-8af0-8d6f698e678a admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:13.285627 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9e8948-1366-45ea-b627-3343f2e09c77 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:13.293412 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac2d28-ec9c-4030-8af0-8d6f698e678a admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:13.293548 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d055d1-3698-406a-85e7-c0bb39cc48d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:13.293548 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d055d1-3698-406a-85e7-c0bb39cc48d5 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:13.293548 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d055d1-3698-406a-85e7-c0bb39cc48d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:13.293548 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d055d1-3698-406a-85e7-c0bb39cc48d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:13.293899 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4814f772-0745-4fb8-8436-8e0261579b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:13.294009 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d055d1-3698-406a-85e7-c0bb39cc48d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:13.294102 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814f772-0745-4fb8-8436-8e0261579b29 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:13.294102 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814f772-0745-4fb8-8436-8e0261579b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:13.294102 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814f772-0745-4fb8-8436-8e0261579b29 admin admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:13.294102 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4814f772-0745-4fb8-8436-8e0261579b29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:13.294456 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d055d1-3698-406a-85e7-c0bb39cc48d5 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:13.295466 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4814f772-0745-4fb8-8436-8e0261579b29 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:13.304148 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 805/1593] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:48:13.305360 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d055d1-3698-406a-85e7-c0bb39cc48d5 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:13.315064 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d055d1-3698-406a-85e7-c0bb39cc48d5 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:13.319842 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15976f91-dd8a-474d-a783-a1efc264cc14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:13.320483 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15976f91-dd8a-474d-a783-a1efc264cc14 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:13.320920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15976f91-dd8a-474d-a783-a1efc264cc14 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:13.321106 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15976f91-dd8a-474d-a783-a1efc264cc14 admin admin] PATH_INFO: `/v3/projects/cedceef6152643f7875eff039186cd55/users/d43636bf86e441b5b72d00f85e01eac7/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:13.336610 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15976f91-dd8a-474d-a783-a1efc264cc14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cedceef6152643f7875eff039186cd55, user_id=d43636bf86e441b5b72d00f85e01eac7, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:13.339312 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15976f91-dd8a-474d-a783-a1efc264cc14 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:13.399336 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 789/1594] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:48:13 2022] POST /identity/v3/projects => generated 381 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:48:13.400224 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4814f772-0745-4fb8-8436-8e0261579b29 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:13.461010 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 806/1595] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:48:13 2022] PUT /identity/v3/projects/cedceef6152643f7875eff039186cd55/users/d43636bf86e441b5b72d00f85e01eac7/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:48:13.461438 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4814f772-0745-4fb8-8436-8e0261579b29 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:13.461789 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15976f91-dd8a-474d-a783-a1efc264cc14 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:13.524657 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd439f-80cc-438c-a5bb-7a21f145fb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:13.525758 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd439f-80cc-438c-a5bb-7a21f145fb27 admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:13.526228 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd439f-80cc-438c-a5bb-7a21f145fb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:13.526738 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd439f-80cc-438c-a5bb-7a21f145fb27 admin admin] PATH_INFO: `/v3/projects/68482ab19f6a4f068555b0762ba4ffce/users/4971a195508f400998673f7cf49d54ac/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:13.530823 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15976f91-dd8a-474d-a783-a1efc264cc14 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:13.536426 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349585ba-05e6-468b-8cb6-bcce658612e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:13.537147 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349585ba-05e6-468b-8cb6-bcce658612e3 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:13.538049 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349585ba-05e6-468b-8cb6-bcce658612e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:13.538366 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349585ba-05e6-468b-8cb6-bcce658612e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:13.539525 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349585ba-05e6-468b-8cb6-bcce658612e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:13.541295 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349585ba-05e6-468b-8cb6-bcce658612e3 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:13.544490 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bd439f-80cc-438c-a5bb-7a21f145fb27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68482ab19f6a4f068555b0762ba4ffce, user_id=4971a195508f400998673f7cf49d54ac, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:13.545559 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: WARNING py.warnings [None req-94bd439f-80cc-438c-a5bb-7a21f145fb27 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 22:48:13.545559 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: warnings.warn(msg) Aug 29 22:48:13.545559 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: Aug 29 22:48:13.551451 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bd439f-80cc-438c-a5bb-7a21f145fb27 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:13.556309 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 807/1596] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:48:13.556309 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349585ba-05e6-468b-8cb6-bcce658612e3 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:13.569050 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349585ba-05e6-468b-8cb6-bcce658612e3 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:13.573649 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b969e-3b77-425d-b43d-6d9a6bf9d626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:13.577937 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b969e-3b77-425d-b43d-6d9a6bf9d626 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:13.577937 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b969e-3b77-425d-b43d-6d9a6bf9d626 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:13.577937 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b969e-3b77-425d-b43d-6d9a6bf9d626 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:13.577937 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555b969e-3b77-425d-b43d-6d9a6bf9d626 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:13.578282 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555b969e-3b77-425d-b43d-6d9a6bf9d626 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:13.588453 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 808/1597] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:48:13.589290 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b969e-3b77-425d-b43d-6d9a6bf9d626 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:13.599230 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b969e-3b77-425d-b43d-6d9a6bf9d626 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:13.603750 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d956652-d15a-452f-94d1-115320b6c02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:13.604330 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d956652-d15a-452f-94d1-115320b6c02e admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:13.604605 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d956652-d15a-452f-94d1-115320b6c02e admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:13.604943 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d956652-d15a-452f-94d1-115320b6c02e admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:13.605581 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d956652-d15a-452f-94d1-115320b6c02e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:13.607134 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d956652-d15a-452f-94d1-115320b6c02e admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:13.617601 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 809/1598] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:48:13.618462 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d956652-d15a-452f-94d1-115320b6c02e admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:13.628258 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d956652-d15a-452f-94d1-115320b6c02e admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:13.632775 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596dd6ab-399a-4236-9e64-c863e8355884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:13.633345 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596dd6ab-399a-4236-9e64-c863e8355884 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:13.633622 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596dd6ab-399a-4236-9e64-c863e8355884 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:13.633892 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596dd6ab-399a-4236-9e64-c863e8355884 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:13.634508 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596dd6ab-399a-4236-9e64-c863e8355884 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:13.636107 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596dd6ab-399a-4236-9e64-c863e8355884 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:13.646457 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 810/1599] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:48:13.647230 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596dd6ab-399a-4236-9e64-c863e8355884 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:13.656449 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596dd6ab-399a-4236-9e64-c863e8355884 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:13.661309 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0847e3db-d9fa-4133-b52c-be8a6f2c8f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:13.661940 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847e3db-d9fa-4133-b52c-be8a6f2c8f0b admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:13.662219 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847e3db-d9fa-4133-b52c-be8a6f2c8f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:13.662486 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847e3db-d9fa-4133-b52c-be8a6f2c8f0b admin admin] PATH_INFO: `/v3/projects/5ac28f3dbe08432e9ce0ce4e9da8b486/users/d0bd8dba03a4498fa2918120389aa1cc/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:13.666776 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 790/1600] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:48:13 2022] PUT /identity/v3/projects/68482ab19f6a4f068555b0762ba4ffce/users/4971a195508f400998673f7cf49d54ac/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:48:13.667687 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd439f-80cc-438c-a5bb-7a21f145fb27 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:13.684473 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0847e3db-d9fa-4133-b52c-be8a6f2c8f0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ac28f3dbe08432e9ce0ce4e9da8b486, user_id=d0bd8dba03a4498fa2918120389aa1cc, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:13.688048 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0847e3db-d9fa-4133-b52c-be8a6f2c8f0b admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:13.730711 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd439f-80cc-438c-a5bb-7a21f145fb27 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:13.735389 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc896a5-da9c-484f-831d-d1432d231b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:13.735989 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc896a5-da9c-484f-831d-d1432d231b1b admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:13.736290 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc896a5-da9c-484f-831d-d1432d231b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:13.736559 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc896a5-da9c-484f-831d-d1432d231b1b admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:13.737213 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc896a5-da9c-484f-831d-d1432d231b1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:13.739162 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc896a5-da9c-484f-831d-d1432d231b1b admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:13.746324 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 811/1601] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:48:13 2022] PUT /identity/v3/projects/5ac28f3dbe08432e9ce0ce4e9da8b486/users/d0bd8dba03a4498fa2918120389aa1cc/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:48:13.747535 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc89d5-d24a-47c4-9c38-5f065e67219a None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:13.747913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc89d5-d24a-47c4-9c38-5f065e67219a None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:13.748139 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc89d5-d24a-47c4-9c38-5f065e67219a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:13.762875 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:13.770733 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-88dc89d5-d24a-47c4-9c38-5f065e67219a None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:13.771980 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-88dc89d5-d24a-47c4-9c38-5f065e67219a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:13.829048 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:13.833675 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 812/1602] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:13.837072 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dc89d5-d24a-47c4-9c38-5f065e67219a None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:13.837182 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 791/1603] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:48:13 2022] POST /identity/v3/users => generated 297 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:48:13.837182 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc896a5-da9c-484f-831d-d1432d231b1b admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:13.843911 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dc89d5-d24a-47c4-9c38-5f065e67219a None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:13.848809 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa10bdd-28ca-46be-9309-3a887e98382e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:13.849470 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa10bdd-28ca-46be-9309-3a887e98382e admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:13.849748 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa10bdd-28ca-46be-9309-3a887e98382e admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:13.850019 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa10bdd-28ca-46be-9309-3a887e98382e admin admin] PATH_INFO: `/v3/projects/d106ca51e01f439f82fd71df4dc2ed71/users/bcd83a222eb04368a409db06389f1b7d/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:13.853365 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc896a5-da9c-484f-831d-d1432d231b1b admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:13.858129 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd8688-d82a-4c65-9988-ad12abeced3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:13.858809 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd8688-d82a-4c65-9988-ad12abeced3c admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:13.862406 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd8688-d82a-4c65-9988-ad12abeced3c admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:13.862406 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd8688-d82a-4c65-9988-ad12abeced3c admin admin] PATH_INFO: `/v3/projects/91c111b2b38c4938ac717066e64a7262/users/3a5e59d1799a4b0ba6dda59d77e91256/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:13.867232 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa10bdd-28ca-46be-9309-3a887e98382e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d106ca51e01f439f82fd71df4dc2ed71, user_id=bcd83a222eb04368a409db06389f1b7d, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:13.870006 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa10bdd-28ca-46be-9309-3a887e98382e admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:13.875347 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acd8688-d82a-4c65-9988-ad12abeced3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=91c111b2b38c4938ac717066e64a7262, user_id=3a5e59d1799a4b0ba6dda59d77e91256, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:13.878086 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acd8688-d82a-4c65-9988-ad12abeced3c admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:13.957916 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 813/1604] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:48:13 2022] PUT /identity/v3/projects/d106ca51e01f439f82fd71df4dc2ed71/users/bcd83a222eb04368a409db06389f1b7d/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:48:13.958842 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa10bdd-28ca-46be-9309-3a887e98382e admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:13.969513 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 792/1605] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:48:13 2022] PUT /identity/v3/projects/91c111b2b38c4938ac717066e64a7262/users/3a5e59d1799a4b0ba6dda59d77e91256/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:48:13.970613 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd8688-d82a-4c65-9988-ad12abeced3c admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:14.034041 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa10bdd-28ca-46be-9309-3a887e98382e admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:14.035771 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd8688-d82a-4c65-9988-ad12abeced3c admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:14.038632 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b8385a-e4b7-4917-8b20-249bed1813b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:14.039288 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8385a-e4b7-4917-8b20-249bed1813b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:14.039565 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8385a-e4b7-4917-8b20-249bed1813b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:14.039843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8385a-e4b7-4917-8b20-249bed1813b6 admin admin] PATH_INFO: `/v3/projects/a0e8b532701e43a39a0fd260e3e5d12d/users/38a89946e1fb4b43aed992078dba0b4f/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:14.040521 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24c3b37-8f7c-4854-a08d-5f3b28f5c8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:14.041205 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c3b37-8f7c-4854-a08d-5f3b28f5c8f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:14.041486 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c3b37-8f7c-4854-a08d-5f3b28f5c8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:14.041759 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c3b37-8f7c-4854-a08d-5f3b28f5c8f7 admin admin] PATH_INFO: `/v3/projects/52aad64da33c4a48aa9eb7e325c5669b/users/53f9f8eb7a20431ca0d679af45946bce/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:14.054374 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b8385a-e4b7-4917-8b20-249bed1813b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0e8b532701e43a39a0fd260e3e5d12d, user_id=38a89946e1fb4b43aed992078dba0b4f, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:14.056310 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24c3b37-8f7c-4854-a08d-5f3b28f5c8f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52aad64da33c4a48aa9eb7e325c5669b, user_id=53f9f8eb7a20431ca0d679af45946bce, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:14.057196 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b8385a-e4b7-4917-8b20-249bed1813b6 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:14.059114 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24c3b37-8f7c-4854-a08d-5f3b28f5c8f7 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:14.179396 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 814/1606] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:48:13 2022] PUT /identity/v3/projects/a0e8b532701e43a39a0fd260e3e5d12d/users/38a89946e1fb4b43aed992078dba0b4f/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:48:14.180747 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f94dac-b8db-4666-836e-9263310e8c88 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:14.181042 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f94dac-b8db-4666-836e-9263310e8c88 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:14.181320 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f94dac-b8db-4666-836e-9263310e8c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:14.210904 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-f1f94dac-b8db-4666-836e-9263310e8c88 None None] MFA Rules not processed for user `4971a195508f400998673f7cf49d54ac`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:14.212257 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-f1f94dac-b8db-4666-836e-9263310e8c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:14.227111 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 793/1607] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:48:13 2022] PUT /identity/v3/projects/52aad64da33c4a48aa9eb7e325c5669b/users/53f9f8eb7a20431ca0d679af45946bce/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:48:14.232768 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e0c5b9-7232-45b4-b870-e60de02c9147 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:14.233504 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e0c5b9-7232-45b4-b870-e60de02c9147 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:14.233811 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e0c5b9-7232-45b4-b870-e60de02c9147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:14.262505 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-c5e0c5b9-7232-45b4-b870-e60de02c9147 None None] MFA Rules not processed for user `d0bd8dba03a4498fa2918120389aa1cc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:14.263893 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-c5e0c5b9-7232-45b4-b870-e60de02c9147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:14.274157 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:14.292014 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 815/1608] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:14.292014 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f94dac-b8db-4666-836e-9263310e8c88 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:14.339409 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 794/1609] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:14.340255 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e0c5b9-7232-45b4-b870-e60de02c9147 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:14.341586 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-c5e0c5b9-7232-45b4-b870-e60de02c9147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:14.356644 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f94dac-b8db-4666-836e-9263310e8c88 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:14.401161 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:14.405949 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e0c5b9-7232-45b4-b870-e60de02c9147 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:14.410820 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b902ec-3b1d-43d6-8657-aa3989f9d165 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:14.422364 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b902ec-3b1d-43d6-8657-aa3989f9d165 service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:14.422364 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b902ec-3b1d-43d6-8657-aa3989f9d165 service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:14.422364 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b902ec-3b1d-43d6-8657-aa3989f9d165 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:14.422813 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1e679-565c-4179-83cb-be8f135b6841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:14.422813 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1e679-565c-4179-83cb-be8f135b6841 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:14.422813 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1e679-565c-4179-83cb-be8f135b6841 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:14.422813 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1e679-565c-4179-83cb-be8f135b6841 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:14.423422 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c1e679-565c-4179-83cb-be8f135b6841 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:14.425387 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c1e679-565c-4179-83cb-be8f135b6841 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:14.439232 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 816/1610] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:48:14.439232 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a49616-f355-46ae-b43b-918761f81ba8 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:14.439232 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a49616-f355-46ae-b43b-918761f81ba8 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:14.439232 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a49616-f355-46ae-b43b-918761f81ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:14.459662 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-b7a49616-f355-46ae-b43b-918761f81ba8 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:14.460889 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-b7a49616-f355-46ae-b43b-918761f81ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:14.470582 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 817/1611] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:14.472312 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d056e75-306d-4778-ab1e-097591f7eeac None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:14.472814 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d056e75-306d-4778-ab1e-097591f7eeac None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:14.473239 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d056e75-306d-4778-ab1e-097591f7eeac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:14.479258 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b902ec-3b1d-43d6-8657-aa3989f9d165 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:14.480537 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b902ec-3b1d-43d6-8657-aa3989f9d165 service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:14.491516 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 795/1612] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:48:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:48:14.492925 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d108a00-08a0-4b87-bedb-565163ee8588 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:14.493187 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d108a00-08a0-4b87-bedb-565163ee8588 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:14.493464 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d108a00-08a0-4b87-bedb-565163ee8588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:14.503813 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-4d056e75-306d-4778-ab1e-097591f7eeac None None] MFA Rules not processed for user `3a5e59d1799a4b0ba6dda59d77e91256`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:14.508756 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-4d056e75-306d-4778-ab1e-097591f7eeac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:14.514719 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-6d108a00-08a0-4b87-bedb-565163ee8588 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:14.516685 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-6d108a00-08a0-4b87-bedb-565163ee8588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:14.524750 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 796/1613] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:14.526096 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f31cf-97d6-4dc8-acd3-74b1b00c4e9e None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:14.526381 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f31cf-97d6-4dc8-acd3-74b1b00c4e9e None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:14.526678 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f31cf-97d6-4dc8-acd3-74b1b00c4e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:14.559356 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-789f31cf-97d6-4dc8-acd3-74b1b00c4e9e None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:14.559356 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-789f31cf-97d6-4dc8-acd3-74b1b00c4e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:14.566975 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 797/1614] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:14.566975 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f31cf-97d6-4dc8-acd3-74b1b00c4e9e None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:14.579123 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f31cf-97d6-4dc8-acd3-74b1b00c4e9e None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:14.584139 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 818/1615] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:14.584616 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b300e3-6b91-4cce-9f71-0780d1b8dd1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:14.585328 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b300e3-6b91-4cce-9f71-0780d1b8dd1d admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:14.585683 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b300e3-6b91-4cce-9f71-0780d1b8dd1d admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:14.586026 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b300e3-6b91-4cce-9f71-0780d1b8dd1d admin admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:14.586528 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2e729-6f57-4fbf-a79a-1139fd9a793d None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:14.586771 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2e729-6f57-4fbf-a79a-1139fd9a793d None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:14.586879 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b300e3-6b91-4cce-9f71-0780d1b8dd1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:14.587061 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2e729-6f57-4fbf-a79a-1139fd9a793d None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:14.588433 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 819/1616] 10.222.0.52 () {54 vars in 962 bytes} [Mon Aug 29 22:48:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:48:14.589151 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b300e3-6b91-4cce-9f71-0780d1b8dd1d admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:14.593772 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec2e729-6f57-4fbf-a79a-1139fd9a793d None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:14.613182 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec2e729-6f57-4fbf-a79a-1139fd9a793d None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:14.618354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3cf12b-270e-4a91-8b77-e0ac7a6982c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:14.618881 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3cf12b-270e-4a91-8b77-e0ac7a6982c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:14.619232 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3cf12b-270e-4a91-8b77-e0ac7a6982c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:14.619448 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3cf12b-270e-4a91-8b77-e0ac7a6982c5 admin admin] PATH_INFO: `/v3/projects/7a82cd47e2cf488b9a81dc73eada77ef/users/2528fff264524ad9b4155c5812c42e80/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:14.637070 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3cf12b-270e-4a91-8b77-e0ac7a6982c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a82cd47e2cf488b9a81dc73eada77ef, user_id=2528fff264524ad9b4155c5812c42e80, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:14.640464 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3cf12b-270e-4a91-8b77-e0ac7a6982c5 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:14.696320 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 798/1617] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:48:14 2022] POST /identity/v3/projects => generated 363 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:48:14.697197 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b300e3-6b91-4cce-9f71-0780d1b8dd1d admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:14.751892 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b300e3-6b91-4cce-9f71-0780d1b8dd1d admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:14.756607 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe045fb1-86fe-4178-b32e-a7b655727daa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:14.757245 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe045fb1-86fe-4178-b32e-a7b655727daa service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:14.757678 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe045fb1-86fe-4178-b32e-a7b655727daa service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:14.757808 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe045fb1-86fe-4178-b32e-a7b655727daa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:14.757941 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:14.763588 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 820/1618] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:48:14 2022] PUT /identity/v3/projects/7a82cd47e2cf488b9a81dc73eada77ef/users/2528fff264524ad9b4155c5812c42e80/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:48:14.764390 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3cf12b-270e-4a91-8b77-e0ac7a6982c5 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:14.833322 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe045fb1-86fe-4178-b32e-a7b655727daa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:14.839385 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe045fb1-86fe-4178-b32e-a7b655727daa service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:14.839562 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3cf12b-270e-4a91-8b77-e0ac7a6982c5 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:14.844345 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61ae13b-6173-4eeb-bd14-a69e5c92e28e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:14.845157 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ae13b-6173-4eeb-bd14-a69e5c92e28e service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:14.846310 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ae13b-6173-4eeb-bd14-a69e5c92e28e service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:14.846310 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ae13b-6173-4eeb-bd14-a69e5c92e28e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:14.852165 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 799/1619] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:48:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:48:14.853406 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe045fb1-86fe-4178-b32e-a7b655727daa service neutron] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:14.862585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe045fb1-86fe-4178-b32e-a7b655727daa service neutron] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:14.866793 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06a6034-9ebc-4b50-baad-7603fb851be2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:14.867515 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a6034-9ebc-4b50-baad-7603fb851be2 service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:14.867955 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a6034-9ebc-4b50-baad-7603fb851be2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:14.868333 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a6034-9ebc-4b50-baad-7603fb851be2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:14.879163 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:14.918635 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61ae13b-6173-4eeb-bd14-a69e5c92e28e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:14.920550 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61ae13b-6173-4eeb-bd14-a69e5c92e28e service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:14.933368 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 821/1620] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:48:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:48:14.934674 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5b9bb-2926-4766-983e-7dd2257ed734 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:14.934966 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5b9bb-2926-4766-983e-7dd2257ed734 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:14.935237 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5b9bb-2926-4766-983e-7dd2257ed734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:14.942082 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06a6034-9ebc-4b50-baad-7603fb851be2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:14.943729 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06a6034-9ebc-4b50-baad-7603fb851be2 service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:14.954922 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 800/1621] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:48:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:48:14.956335 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6ec19-e5e6-41d8-8753-1828452ea7be None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:14.956619 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6ec19-e5e6-41d8-8753-1828452ea7be None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:14.956934 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6ec19-e5e6-41d8-8753-1828452ea7be None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:14.958385 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 801/1622] 10.222.0.52 () {54 vars in 962 bytes} [Mon Aug 29 22:48:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:48:14.958979 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6ec19-e5e6-41d8-8753-1828452ea7be None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:14.971781 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6ec19-e5e6-41d8-8753-1828452ea7be None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:14.976983 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d688eb-6087-4d50-9daf-47179f10fa71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:14.978340 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d688eb-6087-4d50-9daf-47179f10fa71 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:14.978340 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d688eb-6087-4d50-9daf-47179f10fa71 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:14.978340 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d688eb-6087-4d50-9daf-47179f10fa71 admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:14.978692 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-abc5b9bb-2926-4766-983e-7dd2257ed734 None None] MFA Rules not processed for user `bdbf961e02934e88af3a72d6218f32fa`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:14.979320 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d688eb-6087-4d50-9daf-47179f10fa71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:14.979454 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-abc5b9bb-2926-4766-983e-7dd2257ed734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:14.981069 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d688eb-6087-4d50-9daf-47179f10fa71 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:15.054027 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 822/1623] 10.222.0.52 () {58 vars in 1059 bytes} [Mon Aug 29 22:48:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:15.057545 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc5b9bb-2926-4766-983e-7dd2257ed734 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:15.070817 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 802/1624] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:48:14 2022] POST /identity/v3/users => generated 288 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:48:15.070817 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b532433d-3252-4484-a03c-077f6c6d3803 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:15.070817 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b532433d-3252-4484-a03c-077f6c6d3803 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:15.070817 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b532433d-3252-4484-a03c-077f6c6d3803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:15.078875 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc5b9bb-2926-4766-983e-7dd2257ed734 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:15.085788 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d3b4df-7e8e-4a1b-8373-e286761683c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:15.087248 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3b4df-7e8e-4a1b-8373-e286761683c3 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:15.087248 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3b4df-7e8e-4a1b-8373-e286761683c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:15.087248 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3b4df-7e8e-4a1b-8373-e286761683c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:15.088047 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d3b4df-7e8e-4a1b-8373-e286761683c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:15.089764 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d3b4df-7e8e-4a1b-8373-e286761683c3 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:15.092623 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-b532433d-3252-4484-a03c-077f6c6d3803 None None] MFA Rules not processed for user `bdbf961e02934e88af3a72d6218f32fa`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:15.094569 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-b532433d-3252-4484-a03c-077f6c6d3803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:15.104931 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 823/1625] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:48:15 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:48:15.104931 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d3b4df-7e8e-4a1b-8373-e286761683c3 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:15.105426 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 803/1626] 10.222.0.52 () {58 vars in 1059 bytes} [Mon Aug 29 22:48:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:15.106491 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b532433d-3252-4484-a03c-077f6c6d3803 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:15.118590 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b532433d-3252-4484-a03c-077f6c6d3803 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:15.120308 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d3b4df-7e8e-4a1b-8373-e286761683c3 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:15.124058 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be1013-1d12-453b-967b-599ba5418f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:15.125282 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be1013-1d12-453b-967b-599ba5418f7c admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:15.125741 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be1013-1d12-453b-967b-599ba5418f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:15.126270 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be1013-1d12-453b-967b-599ba5418f7c admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:15.127957 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57be1013-1d12-453b-967b-599ba5418f7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:15.129102 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c6bc7-f302-4b3f-af36-6759f1e09e83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:15.131131 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c6bc7-f302-4b3f-af36-6759f1e09e83 service nova] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:15.131540 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57be1013-1d12-453b-967b-599ba5418f7c admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:15.131676 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c6bc7-f302-4b3f-af36-6759f1e09e83 service nova] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:15.131954 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c6bc7-f302-4b3f-af36-6759f1e09e83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:15.144276 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 804/1627] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:48:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:48:15.145837 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be1013-1d12-453b-967b-599ba5418f7c admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:15.150698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:15.163466 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be1013-1d12-453b-967b-599ba5418f7c admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:15.166200 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4779be4-a61a-43f2-8e0e-2b716f9fed31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:15.166931 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4779be4-a61a-43f2-8e0e-2b716f9fed31 admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:15.167220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4779be4-a61a-43f2-8e0e-2b716f9fed31 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:15.167657 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4779be4-a61a-43f2-8e0e-2b716f9fed31 admin admin] PATH_INFO: `/v3/projects/7a82cd47e2cf488b9a81dc73eada77ef/users/2528fff264524ad9b4155c5812c42e80/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:15.169834 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4779be4-a61a-43f2-8e0e-2b716f9fed31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a82cd47e2cf488b9a81dc73eada77ef, user_id=2528fff264524ad9b4155c5812c42e80, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:15.173645 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4779be4-a61a-43f2-8e0e-2b716f9fed31 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:15.187565 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 805/1628] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:48:15 2022] PUT /identity/v3/projects/7a82cd47e2cf488b9a81dc73eada77ef/users/2528fff264524ad9b4155c5812c42e80/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:48:15.188601 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4779be4-a61a-43f2-8e0e-2b716f9fed31 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:15.222225 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52c6bc7-f302-4b3f-af36-6759f1e09e83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:15.223839 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52c6bc7-f302-4b3f-af36-6759f1e09e83 service nova] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:15.246414 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 824/1629] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:48:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:48:15.248546 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c6bc7-f302-4b3f-af36-6759f1e09e83 service nova] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:15.277272 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c6bc7-f302-4b3f-af36-6759f1e09e83 service nova] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:15.283683 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4779be4-a61a-43f2-8e0e-2b716f9fed31 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:15.292154 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624e827b-8405-4a49-873e-bda30b6c5fa1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:15.292995 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e827b-8405-4a49-873e-bda30b6c5fa1 service nova] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:15.293379 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e827b-8405-4a49-873e-bda30b6c5fa1 service nova] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:15.293939 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e827b-8405-4a49-873e-bda30b6c5fa1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:15.375340 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:15.383915 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624e827b-8405-4a49-873e-bda30b6c5fa1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:15.388180 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624e827b-8405-4a49-873e-bda30b6c5fa1 service nova] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:15.402664 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 806/1630] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:48:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:48:15.404304 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f847c-a9b1-41ab-9ecf-515ab0956689 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:15.404685 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f847c-a9b1-41ab-9ecf-515ab0956689 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:15.405158 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f847c-a9b1-41ab-9ecf-515ab0956689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:15.433734 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b3ec2f-7224-47a0-af1c-5dd2cda4a48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:15.434404 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b3ec2f-7224-47a0-af1c-5dd2cda4a48a admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:15.434676 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b3ec2f-7224-47a0-af1c-5dd2cda4a48a admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:15.434950 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b3ec2f-7224-47a0-af1c-5dd2cda4a48a admin admin] PATH_INFO: `/v3/projects/00a715aebe514121866463e092d66fa5/users/11328985ad094ae6840f564834710f16/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:15.453144 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b3ec2f-7224-47a0-af1c-5dd2cda4a48a admin admin] RBAC: Authorizing `identity:create_grant(project_id=00a715aebe514121866463e092d66fa5, user_id=11328985ad094ae6840f564834710f16, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:15.457244 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b3ec2f-7224-47a0-af1c-5dd2cda4a48a admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:15.459162 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-f87f847c-a9b1-41ab-9ecf-515ab0956689 None None] MFA Rules not processed for user `2528fff264524ad9b4155c5812c42e80`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:15.459681 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-f87f847c-a9b1-41ab-9ecf-515ab0956689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:15.540513 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 825/1631] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:48:15 2022] PUT /identity/v3/projects/00a715aebe514121866463e092d66fa5/users/11328985ad094ae6840f564834710f16/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:48:15.547635 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b3ec2f-7224-47a0-af1c-5dd2cda4a48a admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:15.607300 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 807/1632] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:15 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:15.621114 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87f847c-a9b1-41ab-9ecf-515ab0956689 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:15.671935 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b3ec2f-7224-47a0-af1c-5dd2cda4a48a admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:15.677301 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eb77b4-bfc7-45ff-acf6-9e2a84ea9722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:15.678769 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb77b4-bfc7-45ff-acf6-9e2a84ea9722 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:15.678769 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb77b4-bfc7-45ff-acf6-9e2a84ea9722 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:15.678769 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb77b4-bfc7-45ff-acf6-9e2a84ea9722 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:15.679166 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eb77b4-bfc7-45ff-acf6-9e2a84ea9722 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:15.680877 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eb77b4-bfc7-45ff-acf6-9e2a84ea9722 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:15.716827 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 826/1633] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:48:15 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:48:15.719770 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569bf372-b682-406c-a396-03f01217917b None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:15.720321 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569bf372-b682-406c-a396-03f01217917b None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:15.721200 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569bf372-b682-406c-a396-03f01217917b None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:15.723785 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 827/1634] 10.222.0.52 () {54 vars in 912 bytes} [Mon Aug 29 22:48:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:48:15.724788 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569bf372-b682-406c-a396-03f01217917b None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:15.744347 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87f847c-a9b1-41ab-9ecf-515ab0956689 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:15.755038 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569bf372-b682-406c-a396-03f01217917b None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:15.755038 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff609ac0-a03d-4ef4-b11f-ffab546d2d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:15.755038 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff609ac0-a03d-4ef4-b11f-ffab546d2d50 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:15.755038 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff609ac0-a03d-4ef4-b11f-ffab546d2d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:15.755563 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff609ac0-a03d-4ef4-b11f-ffab546d2d50 admin admin] PATH_INFO: `/v3/projects/00a715aebe514121866463e092d66fa5/users/11328985ad094ae6840f564834710f16/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:15.758145 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff609ac0-a03d-4ef4-b11f-ffab546d2d50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00a715aebe514121866463e092d66fa5, user_id=11328985ad094ae6840f564834710f16, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:15.762026 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff609ac0-a03d-4ef4-b11f-ffab546d2d50 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:15.797792 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 828/1635] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:48:15 2022] PUT /identity/v3/projects/00a715aebe514121866463e092d66fa5/users/11328985ad094ae6840f564834710f16/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:48:15.799329 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d15554-07a5-44b7-8828-6681d724d2e6 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:15.799688 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d15554-07a5-44b7-8828-6681d724d2e6 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:15.800064 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d15554-07a5-44b7-8828-6681d724d2e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:15.821205 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9331966-f0a1-4932-b0e6-126ae9b0fc4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:15.822412 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9331966-f0a1-4932-b0e6-126ae9b0fc4b service cinder] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:15.823544 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9331966-f0a1-4932-b0e6-126ae9b0fc4b service cinder] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:15.823831 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9331966-f0a1-4932-b0e6-126ae9b0fc4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:15.825665 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-c6d15554-07a5-44b7-8828-6681d724d2e6 None None] MFA Rules not processed for user `bdbf961e02934e88af3a72d6218f32fa`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:15.828043 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-c6d15554-07a5-44b7-8828-6681d724d2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:15.853341 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:15.916000 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:15.927578 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 829/1636] 10.222.0.52 () {58 vars in 1009 bytes} [Mon Aug 29 22:48:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:15.929077 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7ce97-e579-434b-9198-0907a7e6a24c None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:15.929368 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7ce97-e579-434b-9198-0907a7e6a24c None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:15.932804 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7ce97-e579-434b-9198-0907a7e6a24c None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:15.936080 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 830/1637] 10.222.0.52 () {54 vars in 912 bytes} [Mon Aug 29 22:48:15 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:48:15.939121 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a7723-5237-4b41-b8fb-1985e9529164 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:15.939415 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a7723-5237-4b41-b8fb-1985e9529164 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:15.939683 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a7723-5237-4b41-b8fb-1985e9529164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:15.953700 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9331966-f0a1-4932-b0e6-126ae9b0fc4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:15.956684 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9331966-f0a1-4932-b0e6-126ae9b0fc4b service cinder] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:15.986021 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 808/1638] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:48:15 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:48:15.987590 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b92851-a4ac-4484-8e0d-17e24668a1a4 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:15.987950 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b92851-a4ac-4484-8e0d-17e24668a1a4 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:15.988286 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b92851-a4ac-4484-8e0d-17e24668a1a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:15.989916 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-7c4a7723-5237-4b41-b8fb-1985e9529164 None None] MFA Rules not processed for user `11328985ad094ae6840f564834710f16`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:15.991846 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-7c4a7723-5237-4b41-b8fb-1985e9529164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:16.024036 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-58b92851-a4ac-4484-8e0d-17e24668a1a4 None None] MFA Rules not processed for user `bdbf961e02934e88af3a72d6218f32fa`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:16.025369 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-58b92851-a4ac-4484-8e0d-17e24668a1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:16.037996 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 809/1639] 10.222.0.52 () {58 vars in 1009 bytes} [Mon Aug 29 22:48:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:16.044344 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b92851-a4ac-4484-8e0d-17e24668a1a4 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:16.102986 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 831/1640] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:15 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:16.103697 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a7723-5237-4b41-b8fb-1985e9529164 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:16.119858 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b92851-a4ac-4484-8e0d-17e24668a1a4 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:16.120644 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a7723-5237-4b41-b8fb-1985e9529164 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:16.125374 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f861d6f-0f47-4821-ac80-af032224c954 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:16.126011 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd96986b-6d05-4406-980b-70836c9c8669 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:16.126419 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f861d6f-0f47-4821-ac80-af032224c954 service cinder] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:16.126925 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd96986b-6d05-4406-980b-70836c9c8669 service cinder] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:16.127159 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f861d6f-0f47-4821-ac80-af032224c954 service cinder] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:16.127574 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f861d6f-0f47-4821-ac80-af032224c954 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:16.127988 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd96986b-6d05-4406-980b-70836c9c8669 service cinder] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:16.129109 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd96986b-6d05-4406-980b-70836c9c8669 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:16.142255 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f861d6f-0f47-4821-ac80-af032224c954 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:16.143913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f861d6f-0f47-4821-ac80-af032224c954 service cinder] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:16.144620 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd96986b-6d05-4406-980b-70836c9c8669 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:16.146352 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd96986b-6d05-4406-980b-70836c9c8669 service cinder] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:16.164818 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 810/1642] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:48:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:48:16.174161 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 832/1642] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:48:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:48:17.627277 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd96986b-6d05-4406-980b-70836c9c8669 service cinder] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:17.636971 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd96986b-6d05-4406-980b-70836c9c8669 service cinder] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:17.658395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5909e-716a-470e-8ee9-42dc3ac684d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:17.659165 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5909e-716a-470e-8ee9-42dc3ac684d2 service cinder] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:17.659549 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5909e-716a-470e-8ee9-42dc3ac684d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:17.659928 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5909e-716a-470e-8ee9-42dc3ac684d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:17.675907 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b5909e-716a-470e-8ee9-42dc3ac684d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:17.682316 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b5909e-716a-470e-8ee9-42dc3ac684d2 service cinder] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:17.705033 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 811/1643] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:48:17 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:48:17.806387 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f861d6f-0f47-4821-ac80-af032224c954 service cinder] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:17.817629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f861d6f-0f47-4821-ac80-af032224c954 service cinder] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:17.829091 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f852d676-fe9a-4641-a548-2ee401d81d0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:17.829717 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852d676-fe9a-4641-a548-2ee401d81d0f service cinder] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:17.830005 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852d676-fe9a-4641-a548-2ee401d81d0f service cinder] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:17.830280 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852d676-fe9a-4641-a548-2ee401d81d0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:17.966790 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f852d676-fe9a-4641-a548-2ee401d81d0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:17.973980 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f852d676-fe9a-4641-a548-2ee401d81d0f service cinder] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:18.006895 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 833/1644] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:48:17 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:48:18.199145 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5909e-716a-470e-8ee9-42dc3ac684d2 service cinder] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:18.210662 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5909e-716a-470e-8ee9-42dc3ac684d2 service cinder] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:18.215581 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46724d12-e014-4f12-94bd-62064b7e9c25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:18.216114 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46724d12-e014-4f12-94bd-62064b7e9c25 service cinder] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:18.216494 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46724d12-e014-4f12-94bd-62064b7e9c25 service cinder] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:18.216884 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46724d12-e014-4f12-94bd-62064b7e9c25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:18.339195 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46724d12-e014-4f12-94bd-62064b7e9c25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:18.344260 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46724d12-e014-4f12-94bd-62064b7e9c25 service cinder] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:18.365815 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 812/1645] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:48:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:48:22.337351 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f852d676-fe9a-4641-a548-2ee401d81d0f service cinder] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:22.404070 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:22.562618 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f852d676-fe9a-4641-a548-2ee401d81d0f service cinder] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:22.578317 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca70a9-cba0-422a-ac3a-8a40de83fae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:22.579029 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca70a9-cba0-422a-ac3a-8a40de83fae1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:22.584804 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca70a9-cba0-422a-ac3a-8a40de83fae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:22.584804 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca70a9-cba0-422a-ac3a-8a40de83fae1 admin admin] PATH_INFO: `/v3/users/d0bd8dba03a4498fa2918120389aa1cc` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:22.585555 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ca70a9-cba0-422a-ac3a-8a40de83fae1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0bd8dba03a4498fa2918120389aa1cc)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:22.588225 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ca70a9-cba0-422a-ac3a-8a40de83fae1 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:22.926436 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-63ca70a9-cba0-422a-ac3a-8a40de83fae1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0bd8dba03a4498fa2918120389aa1cc'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:23.043102 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-63ca70a9-cba0-422a-ac3a-8a40de83fae1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0bd8dba03a4498fa2918120389aa1cc'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:23.072488 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-63ca70a9-cba0-422a-ac3a-8a40de83fae1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0bd8dba03a4498fa2918120389aa1cc'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:23.096004 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-63ca70a9-cba0-422a-ac3a-8a40de83fae1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0bd8dba03a4498fa2918120389aa1cc'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:23.097627 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-63ca70a9-cba0-422a-ac3a-8a40de83fae1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0bd8dba03a4498fa2918120389aa1cc'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:23.111303 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 834/1646] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:48:22 2022] DELETE /identity/v3/users/d0bd8dba03a4498fa2918120389aa1cc => generated 0 bytes in 774 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:48:23.115446 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69528abd-4ca9-49ac-9fc9-b8b8b5a090a1 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:23.116274 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69528abd-4ca9-49ac-9fc9-b8b8b5a090a1 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:23.116815 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69528abd-4ca9-49ac-9fc9-b8b8b5a090a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:23.146605 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-69528abd-4ca9-49ac-9fc9-b8b8b5a090a1 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:23.148383 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-69528abd-4ca9-49ac-9fc9-b8b8b5a090a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:23.343738 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 813/1647] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:23.381082 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca70a9-cba0-422a-ac3a-8a40de83fae1 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:23.383285 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-63ca70a9-cba0-422a-ac3a-8a40de83fae1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:23.501187 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca70a9-cba0-422a-ac3a-8a40de83fae1 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:23.506845 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc32389-f40b-40eb-922c-4df821827b61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:23.507627 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc32389-f40b-40eb-922c-4df821827b61 service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:23.507912 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc32389-f40b-40eb-922c-4df821827b61 service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:23.508185 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc32389-f40b-40eb-922c-4df821827b61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:23.551344 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc32389-f40b-40eb-922c-4df821827b61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:23.555404 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc32389-f40b-40eb-922c-4df821827b61 service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:23.586449 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 835/1648] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:48:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:48:24.908845 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09aeb0-e37a-4631-8a65-e75560a23d5c None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:24.913644 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09aeb0-e37a-4631-8a65-e75560a23d5c None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:24.913644 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09aeb0-e37a-4631-8a65-e75560a23d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:24.926383 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:24.996502 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-8b09aeb0-e37a-4631-8a65-e75560a23d5c None None] MFA Rules not processed for user `d43636bf86e441b5b72d00f85e01eac7`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:25.002865 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-8b09aeb0-e37a-4631-8a65-e75560a23d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:25.166876 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 814/1649] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:25.178043 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92f2f4-cb28-431a-87d2-d48e0101f045 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:25.179481 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92f2f4-cb28-431a-87d2-d48e0101f045 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:25.179871 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92f2f4-cb28-431a-87d2-d48e0101f045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:25.220224 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-6f92f2f4-cb28-431a-87d2-d48e0101f045 None None] MFA Rules not processed for user `d43636bf86e441b5b72d00f85e01eac7`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:25.222996 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-6f92f2f4-cb28-431a-87d2-d48e0101f045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:25.244223 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 836/1650] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:48:25.257723 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b09aeb0-e37a-4631-8a65-e75560a23d5c None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:25.259603 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-8b09aeb0-e37a-4631-8a65-e75560a23d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:25.432737 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b09aeb0-e37a-4631-8a65-e75560a23d5c None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:25.458712 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f564ec7-6893-4619-9b51-a678df2f8301 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:25.459906 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f564ec7-6893-4619-9b51-a678df2f8301 service nova] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:25.459906 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f564ec7-6893-4619-9b51-a678df2f8301 service nova] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:25.460081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f564ec7-6893-4619-9b51-a678df2f8301 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:25.471591 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f564ec7-6893-4619-9b51-a678df2f8301 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:25.473274 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f564ec7-6893-4619-9b51-a678df2f8301 service nova] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:25.484119 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 815/1651] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:48:25 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:48:25.639359 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f92f2f4-cb28-431a-87d2-d48e0101f045 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:25.641949 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-6f92f2f4-cb28-431a-87d2-d48e0101f045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:25.695766 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f92f2f4-cb28-431a-87d2-d48e0101f045 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:25.714179 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555d7e06-b289-4344-a325-0d11284a8ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:25.716301 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d7e06-b289-4344-a325-0d11284a8ad3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:25.716837 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d7e06-b289-4344-a325-0d11284a8ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:25.717365 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d7e06-b289-4344-a325-0d11284a8ad3 admin admin] PATH_INFO: `/v3/projects/5ac28f3dbe08432e9ce0ce4e9da8b486` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:25.719837 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555d7e06-b289-4344-a325-0d11284a8ad3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ac28f3dbe08432e9ce0ce4e9da8b486)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:25.721372 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: WARNING py.warnings [None req-555d7e06-b289-4344-a325-0d11284a8ad3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 22:48:25.721372 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: warnings.warn(msg) Aug 29 22:48:25.721372 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: Aug 29 22:48:25.733607 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555d7e06-b289-4344-a325-0d11284a8ad3 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:25.936703 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:26.010125 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-555d7e06-b289-4344-a325-0d11284a8ad3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ac28f3dbe08432e9ce0ce4e9da8b486', 'request_id': 'req-555d7e06-b289-4344-a325-0d11284a8ad3'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:26.026284 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-555d7e06-b289-4344-a325-0d11284a8ad3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ac28f3dbe08432e9ce0ce4e9da8b486', 'request_id': 'req-555d7e06-b289-4344-a325-0d11284a8ad3'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:26.078912 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-555d7e06-b289-4344-a325-0d11284a8ad3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ac28f3dbe08432e9ce0ce4e9da8b486', 'request_id': 'req-555d7e06-b289-4344-a325-0d11284a8ad3'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:26.234432 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-555d7e06-b289-4344-a325-0d11284a8ad3 admin admin] The token cache is being invalidate because project 5ac28f3dbe08432e9ce0ce4e9da8b486 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:48:26.243975 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-555d7e06-b289-4344-a325-0d11284a8ad3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:26.243975 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-555d7e06-b289-4344-a325-0d11284a8ad3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:26.243975 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 837/1652] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:48:25 2022] DELETE /identity/v3/projects/5ac28f3dbe08432e9ce0ce4e9da8b486 => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:48:26.245653 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a4d0d-0c7c-4084-8b8a-e4485c7d76fd None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:26.246116 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a4d0d-0c7c-4084-8b8a-e4485c7d76fd None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:26.246202 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a4d0d-0c7c-4084-8b8a-e4485c7d76fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:26.307998 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-a56a4d0d-0c7c-4084-8b8a-e4485c7d76fd None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:26.309258 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-a56a4d0d-0c7c-4084-8b8a-e4485c7d76fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:26.461266 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 816/1653] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:26.466737 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a9419-95ba-4bea-904a-de80a0af6af1 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:26.467028 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a9419-95ba-4bea-904a-de80a0af6af1 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:26.467310 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a9419-95ba-4bea-904a-de80a0af6af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:26.509875 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-081a9419-95ba-4bea-904a-de80a0af6af1 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:26.511111 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-081a9419-95ba-4bea-904a-de80a0af6af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:26.534209 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 838/1654] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:26.540185 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a4d0d-0c7c-4084-8b8a-e4485c7d76fd None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:26.555171 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a4d0d-0c7c-4084-8b8a-e4485c7d76fd None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:26.567652 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaac0161-6cd1-4b99-b66c-a886935e2fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:26.568513 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac0161-6cd1-4b99-b66c-a886935e2fa8 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:26.572205 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac0161-6cd1-4b99-b66c-a886935e2fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:26.572678 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac0161-6cd1-4b99-b66c-a886935e2fa8 admin admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:26.573797 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaac0161-6cd1-4b99-b66c-a886935e2fa8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:48:26.574265 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaac0161-6cd1-4b99-b66c-a886935e2fa8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:26.579821 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaac0161-6cd1-4b99-b66c-a886935e2fa8 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:26.622129 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 817/1655] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:48:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:48:26.628526 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a9419-95ba-4bea-904a-de80a0af6af1 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:26.655862 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a9419-95ba-4bea-904a-de80a0af6af1 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:26.667663 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb998271-b494-4cf5-98ec-9b6bc0c75c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:26.668357 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb998271-b494-4cf5-98ec-9b6bc0c75c54 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:26.668645 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb998271-b494-4cf5-98ec-9b6bc0c75c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:26.668944 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb998271-b494-4cf5-98ec-9b6bc0c75c54 admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:26.672356 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb998271-b494-4cf5-98ec-9b6bc0c75c54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:26.682075 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb998271-b494-4cf5-98ec-9b6bc0c75c54 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:26.698945 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa871844-7d3a-4872-b6a0-f8889e889dc4 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:26.699586 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa871844-7d3a-4872-b6a0-f8889e889dc4 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:26.699987 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa871844-7d3a-4872-b6a0-f8889e889dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:26.770391 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:26.791545 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-fa871844-7d3a-4872-b6a0-f8889e889dc4 None None] MFA Rules not processed for user `bcd83a222eb04368a409db06389f1b7d`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:26.793056 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-fa871844-7d3a-4872-b6a0-f8889e889dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:26.825590 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 839/1656] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:48:26 2022] POST /identity/v3/projects => generated 373 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:48:26.830850 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb998271-b494-4cf5-98ec-9b6bc0c75c54 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:26.970375 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 818/1657] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:26.988788 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa871844-7d3a-4872-b6a0-f8889e889dc4 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:26.993291 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-fa871844-7d3a-4872-b6a0-f8889e889dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:27.015060 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb998271-b494-4cf5-98ec-9b6bc0c75c54 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:27.031932 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7698971e-dc54-4bd8-88a4-3f2c6c154115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:27.035684 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7698971e-dc54-4bd8-88a4-3f2c6c154115 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:27.035684 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7698971e-dc54-4bd8-88a4-3f2c6c154115 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:27.035684 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7698971e-dc54-4bd8-88a4-3f2c6c154115 admin admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:27.035684 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7698971e-dc54-4bd8-88a4-3f2c6c154115 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:27.036061 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7698971e-dc54-4bd8-88a4-3f2c6c154115 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:27.127499 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa871844-7d3a-4872-b6a0-f8889e889dc4 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:27.148771 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 840/1658] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:48:26 2022] POST /identity/v3/users => generated 287 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:48:27.150193 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f391f3e-193b-4026-b651-702fff399742 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:27.150579 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f391f3e-193b-4026-b651-702fff399742 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:27.150930 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f391f3e-193b-4026-b651-702fff399742 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:27.153611 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 841/1659] 10.222.0.52 () {54 vars in 912 bytes} [Mon Aug 29 22:48:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:48:27.154704 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f391f3e-193b-4026-b651-702fff399742 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:27.201448 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f391f3e-193b-4026-b651-702fff399742 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:27.232368 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d8148-e364-48ce-8ab9-bfd1c1d4449e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:27.235257 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d8148-e364-48ce-8ab9-bfd1c1d4449e admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:27.237997 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d8148-e364-48ce-8ab9-bfd1c1d4449e admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:27.239793 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d8148-e364-48ce-8ab9-bfd1c1d4449e admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:27.242185 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88d8148-e364-48ce-8ab9-bfd1c1d4449e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:27.246074 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88d8148-e364-48ce-8ab9-bfd1c1d4449e admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:27.270984 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 842/1660] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:48:27.275765 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943faafa-94c5-4bba-a8b3-a9b659bbbb6b None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:27.277166 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943faafa-94c5-4bba-a8b3-a9b659bbbb6b None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:27.280003 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943faafa-94c5-4bba-a8b3-a9b659bbbb6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:27.355827 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81938bdd-de74-4e1a-8567-156dc3380bdd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:27.357116 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81938bdd-de74-4e1a-8567-156dc3380bdd service nova] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:27.357541 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81938bdd-de74-4e1a-8567-156dc3380bdd service nova] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:27.357975 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81938bdd-de74-4e1a-8567-156dc3380bdd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:27.375008 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-943faafa-94c5-4bba-a8b3-a9b659bbbb6b None None] MFA Rules not processed for user `0fe5e8c4cb324ffb9d5b80f18d123727`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:27.377269 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-943faafa-94c5-4bba-a8b3-a9b659bbbb6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:27.429641 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:27.465425 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 843/1661] 10.222.0.52 () {58 vars in 1009 bytes} [Mon Aug 29 22:48:27 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:27.466677 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943faafa-94c5-4bba-a8b3-a9b659bbbb6b None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:27.480131 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943faafa-94c5-4bba-a8b3-a9b659bbbb6b None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:27.485018 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81938bdd-de74-4e1a-8567-156dc3380bdd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:27.487092 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81938bdd-de74-4e1a-8567-156dc3380bdd service nova] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:27.493132 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec0cc1-4021-4a47-9435-5b6e983011b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:27.493769 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec0cc1-4021-4a47-9435-5b6e983011b4 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:27.494044 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec0cc1-4021-4a47-9435-5b6e983011b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:27.495504 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec0cc1-4021-4a47-9435-5b6e983011b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:27.496312 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fec0cc1-4021-4a47-9435-5b6e983011b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:27.498756 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fec0cc1-4021-4a47-9435-5b6e983011b4 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:27.501743 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 819/1662] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:48:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:48:27.507671 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:27.520270 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 844/1663] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:48:27.523925 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81938bdd-de74-4e1a-8567-156dc3380bdd service nova] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:27.535571 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec0cc1-4021-4a47-9435-5b6e983011b4 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:27.537927 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-0fec0cc1-4021-4a47-9435-5b6e983011b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:27.543112 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81938bdd-de74-4e1a-8567-156dc3380bdd service nova] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:27.557753 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0446b9c3-5d39-4222-b555-c4c50440ecbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:27.559784 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446b9c3-5d39-4222-b555-c4c50440ecbc admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:27.562721 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446b9c3-5d39-4222-b555-c4c50440ecbc admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:27.565018 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446b9c3-5d39-4222-b555-c4c50440ecbc admin admin] PATH_INFO: `/v3/projects/8bb243f3e77d4496bbe7858259293a44/users/57bd2ba043674ef9b23d0571e65aea4e/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:27.570614 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec0cc1-4021-4a47-9435-5b6e983011b4 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:27.578709 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d98ef25-3142-4006-a3b1-7893b3f7eda8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:27.579551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d98ef25-3142-4006-a3b1-7893b3f7eda8 service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:27.583110 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d98ef25-3142-4006-a3b1-7893b3f7eda8 service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:27.583110 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d98ef25-3142-4006-a3b1-7893b3f7eda8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:27.599686 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d98ef25-3142-4006-a3b1-7893b3f7eda8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:27.601476 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d98ef25-3142-4006-a3b1-7893b3f7eda8 service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:27.615119 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0446b9c3-5d39-4222-b555-c4c50440ecbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bb243f3e77d4496bbe7858259293a44, user_id=57bd2ba043674ef9b23d0571e65aea4e, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:27.620992 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 845/1664] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:48:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:48:27.621494 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0446b9c3-5d39-4222-b555-c4c50440ecbc admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:27.746474 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 820/1665] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:48:27 2022] PUT /identity/v3/projects/8bb243f3e77d4496bbe7858259293a44/users/57bd2ba043674ef9b23d0571e65aea4e/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:48:27.763736 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0275ed06-9d19-4b51-aa4a-16d83f272b51 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:27.764023 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0275ed06-9d19-4b51-aa4a-16d83f272b51 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:27.764311 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0275ed06-9d19-4b51-aa4a-16d83f272b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:27.812360 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-0275ed06-9d19-4b51-aa4a-16d83f272b51 None None] MFA Rules not processed for user `57bd2ba043674ef9b23d0571e65aea4e`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:27.814924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-0275ed06-9d19-4b51-aa4a-16d83f272b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:27.998668 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 846/1666] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:28.339853 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0446b9c3-5d39-4222-b555-c4c50440ecbc admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:28.581390 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0446b9c3-5d39-4222-b555-c4c50440ecbc admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:28.596763 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98dead3-bdac-4d83-b37a-38e826d2f0ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:28.597584 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98dead3-bdac-4d83-b37a-38e826d2f0ae service nova] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:28.597932 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98dead3-bdac-4d83-b37a-38e826d2f0ae service nova] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:28.600160 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98dead3-bdac-4d83-b37a-38e826d2f0ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:28.621303 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98dead3-bdac-4d83-b37a-38e826d2f0ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:28.625704 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98dead3-bdac-4d83-b37a-38e826d2f0ae service nova] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:28.663910 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 821/1667] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:48:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:48:29.111423 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0275ed06-9d19-4b51-aa4a-16d83f272b51 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:29.111423 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-0275ed06-9d19-4b51-aa4a-16d83f272b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:29.160612 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98dead3-bdac-4d83-b37a-38e826d2f0ae service nova] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:29.288249 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98dead3-bdac-4d83-b37a-38e826d2f0ae service nova] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:29.295559 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78747dc8-8a51-4c80-8bb7-eb0dc271b4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:29.298591 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78747dc8-8a51-4c80-8bb7-eb0dc271b4ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:29.298991 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78747dc8-8a51-4c80-8bb7-eb0dc271b4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:29.299334 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0275ed06-9d19-4b51-aa4a-16d83f272b51 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:29.299455 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78747dc8-8a51-4c80-8bb7-eb0dc271b4ca admin admin] PATH_INFO: `/v3/users/57bd2ba043674ef9b23d0571e65aea4e` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:29.300850 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78747dc8-8a51-4c80-8bb7-eb0dc271b4ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=57bd2ba043674ef9b23d0571e65aea4e)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:29.302206 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: WARNING py.warnings [None req-78747dc8-8a51-4c80-8bb7-eb0dc271b4ca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 22:48:29.302206 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: warnings.warn(msg) Aug 29 22:48:29.302206 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: Aug 29 22:48:29.303949 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78747dc8-8a51-4c80-8bb7-eb0dc271b4ca admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:29.348357 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d724fbbd-be79-40d2-b473-ed5b17908314 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8fd0f6e5b144d2ca78b8d96722502a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:29.348357 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724fbbd-be79-40d2-b473-ed5b17908314 service placement] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:29.348357 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724fbbd-be79-40d2-b473-ed5b17908314 service placement] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:29.348357 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724fbbd-be79-40d2-b473-ed5b17908314 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:29.356124 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-d724fbbd-be79-40d2-b473-ed5b17908314 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:29.409522 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d724fbbd-be79-40d2-b473-ed5b17908314 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:29.410514 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d724fbbd-be79-40d2-b473-ed5b17908314 service placement] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:29.472906 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 847/1668] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:48:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:48:29.623821 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:29.642349 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-78747dc8-8a51-4c80-8bb7-eb0dc271b4ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57bd2ba043674ef9b23d0571e65aea4e'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:29.668024 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-78747dc8-8a51-4c80-8bb7-eb0dc271b4ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57bd2ba043674ef9b23d0571e65aea4e'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:29.680164 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-78747dc8-8a51-4c80-8bb7-eb0dc271b4ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57bd2ba043674ef9b23d0571e65aea4e'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:29.756053 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-78747dc8-8a51-4c80-8bb7-eb0dc271b4ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57bd2ba043674ef9b23d0571e65aea4e'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:29.756923 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-78747dc8-8a51-4c80-8bb7-eb0dc271b4ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57bd2ba043674ef9b23d0571e65aea4e'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:29.765385 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 822/1669] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:48:29 2022] DELETE /identity/v3/users/57bd2ba043674ef9b23d0571e65aea4e => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:48:29.770551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6d9cc-f65d-4ff4-8e1c-27ad45d5b353 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:29.771178 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6d9cc-f65d-4ff4-8e1c-27ad45d5b353 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:29.771676 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6d9cc-f65d-4ff4-8e1c-27ad45d5b353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:29.809309 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-6db6d9cc-f65d-4ff4-8e1c-27ad45d5b353 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:29.813753 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-6db6d9cc-f65d-4ff4-8e1c-27ad45d5b353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:30.001798 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 848/1670] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:30.124672 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78747dc8-8a51-4c80-8bb7-eb0dc271b4ca admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:30.126863 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-78747dc8-8a51-4c80-8bb7-eb0dc271b4ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:30.267595 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78747dc8-8a51-4c80-8bb7-eb0dc271b4ca admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:30.297088 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc34e15-eda8-485d-8997-773f8975239c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:30.315287 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc34e15-eda8-485d-8997-773f8975239c service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:30.316963 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc34e15-eda8-485d-8997-773f8975239c service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:30.317256 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc34e15-eda8-485d-8997-773f8975239c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:30.359372 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc34e15-eda8-485d-8997-773f8975239c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:30.368629 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc34e15-eda8-485d-8997-773f8975239c service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:30.402681 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 823/1671] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:48:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:48:31.559013 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c01a0-884f-4ab4-91b8-baeb589ee189 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:31.559560 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c01a0-884f-4ab4-91b8-baeb589ee189 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:31.559560 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c01a0-884f-4ab4-91b8-baeb589ee189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:31.591674 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:31.662579 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-f99c01a0-884f-4ab4-91b8-baeb589ee189 None None] MFA Rules not processed for user `38a89946e1fb4b43aed992078dba0b4f`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:31.665285 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-f99c01a0-884f-4ab4-91b8-baeb589ee189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:31.785791 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 849/1672] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:31 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:31.800437 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc34e15-eda8-485d-8997-773f8975239c service neutron] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:31.801879 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-fcc34e15-eda8-485d-8997-773f8975239c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:31.940417 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5594def6-1eb2-4dd2-890b-3277a5751780 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:31.940706 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5594def6-1eb2-4dd2-890b-3277a5751780 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:31.941013 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5594def6-1eb2-4dd2-890b-3277a5751780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:31.961204 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc34e15-eda8-485d-8997-773f8975239c service neutron] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:31.975305 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709a84b4-2d38-4db5-86f8-beb86a3f9d9c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:31.976391 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a84b4-2d38-4db5-86f8-beb86a3f9d9c service nova] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:31.976919 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a84b4-2d38-4db5-86f8-beb86a3f9d9c service nova] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:31.983730 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a84b4-2d38-4db5-86f8-beb86a3f9d9c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:31.997561 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-5594def6-1eb2-4dd2-890b-3277a5751780 None None] MFA Rules not processed for user `53f9f8eb7a20431ca0d679af45946bce`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:31.998985 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-5594def6-1eb2-4dd2-890b-3277a5751780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:32.010392 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709a84b4-2d38-4db5-86f8-beb86a3f9d9c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:32.010392 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709a84b4-2d38-4db5-86f8-beb86a3f9d9c service nova] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:32.024328 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 824/1673] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:48:31 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:48:32.211020 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 850/1674] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:31 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:48:32.222323 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0cc3cf-3e56-417d-a50b-e4821b630a6f None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:32.222323 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0cc3cf-3e56-417d-a50b-e4821b630a6f None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:32.222576 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0cc3cf-3e56-417d-a50b-e4821b630a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:32.236562 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5594def6-1eb2-4dd2-890b-3277a5751780 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:32.239085 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-5594def6-1eb2-4dd2-890b-3277a5751780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:32.293890 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-7c0cc3cf-3e56-417d-a50b-e4821b630a6f None None] MFA Rules not processed for user `53f9f8eb7a20431ca0d679af45946bce`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:32.294527 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5594def6-1eb2-4dd2-890b-3277a5751780 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:32.298083 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-7c0cc3cf-3e56-417d-a50b-e4821b630a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:32.308606 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019837e-6d53-4ea7-afd2-607806180caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:32.309258 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019837e-6d53-4ea7-afd2-607806180caf admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:32.309542 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019837e-6d53-4ea7-afd2-607806180caf admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:32.314093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019837e-6d53-4ea7-afd2-607806180caf admin admin] PATH_INFO: `/v3/projects/8bb243f3e77d4496bbe7858259293a44` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:32.319245 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a019837e-6d53-4ea7-afd2-607806180caf admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bb243f3e77d4496bbe7858259293a44)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:32.320894 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 825/1675] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:32 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:32.322951 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a019837e-6d53-4ea7-afd2-607806180caf admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:32.333029 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0cc3cf-3e56-417d-a50b-e4821b630a6f None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:32.335385 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-7c0cc3cf-3e56-417d-a50b-e4821b630a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:32.385048 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0cc3cf-3e56-417d-a50b-e4821b630a6f None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:32.392982 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4806c-bccf-4cf0-a8a1-b62713b3a6b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:32.393707 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4806c-bccf-4cf0-a8a1-b62713b3a6b2 service nova] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:32.394223 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4806c-bccf-4cf0-a8a1-b62713b3a6b2 service nova] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:32.394593 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4806c-bccf-4cf0-a8a1-b62713b3a6b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:32.406012 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e4806c-bccf-4cf0-a8a1-b62713b3a6b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:32.410888 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e4806c-bccf-4cf0-a8a1-b62713b3a6b2 service nova] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:32.431107 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 826/1676] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:48:32 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:48:32.509844 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-a019837e-6d53-4ea7-afd2-607806180caf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bb243f3e77d4496bbe7858259293a44', 'request_id': 'req-a019837e-6d53-4ea7-afd2-607806180caf'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:32.521457 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-a019837e-6d53-4ea7-afd2-607806180caf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bb243f3e77d4496bbe7858259293a44', 'request_id': 'req-a019837e-6d53-4ea7-afd2-607806180caf'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:32.552992 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-a019837e-6d53-4ea7-afd2-607806180caf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bb243f3e77d4496bbe7858259293a44', 'request_id': 'req-a019837e-6d53-4ea7-afd2-607806180caf'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:32.650873 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-a019837e-6d53-4ea7-afd2-607806180caf admin admin] The token cache is being invalidate because project 8bb243f3e77d4496bbe7858259293a44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:48:32.651966 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-a019837e-6d53-4ea7-afd2-607806180caf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:32.653064 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-a019837e-6d53-4ea7-afd2-607806180caf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:32.654801 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 851/1677] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:48:32 2022] DELETE /identity/v3/projects/8bb243f3e77d4496bbe7858259293a44 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:48:32.661651 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372885b8-17b3-4838-8602-8fe7e7b27130 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:32.662158 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372885b8-17b3-4838-8602-8fe7e7b27130 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:32.662613 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372885b8-17b3-4838-8602-8fe7e7b27130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:32.722770 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-372885b8-17b3-4838-8602-8fe7e7b27130 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:32.727888 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-372885b8-17b3-4838-8602-8fe7e7b27130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:32.773887 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:32.978648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 827/1678] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:32.988086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70eaef-2f08-4584-b690-3221bf51832d None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:32.988685 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70eaef-2f08-4584-b690-3221bf51832d None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:32.988912 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70eaef-2f08-4584-b690-3221bf51832d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:33.012258 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:33.060382 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-9b70eaef-2f08-4584-b690-3221bf51832d None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:33.062500 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-9b70eaef-2f08-4584-b690-3221bf51832d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:33.091457 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 852/1679] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:33.111057 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372885b8-17b3-4838-8602-8fe7e7b27130 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:33.130945 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372885b8-17b3-4838-8602-8fe7e7b27130 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:33.146283 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecbe2f9-7add-42af-b2ff-b5d5705d9480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:33.146932 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecbe2f9-7add-42af-b2ff-b5d5705d9480 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:33.147222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecbe2f9-7add-42af-b2ff-b5d5705d9480 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:33.147517 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecbe2f9-7add-42af-b2ff-b5d5705d9480 admin admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:33.148178 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecbe2f9-7add-42af-b2ff-b5d5705d9480 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:48:33.148655 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecbe2f9-7add-42af-b2ff-b5d5705d9480 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:33.150471 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecbe2f9-7add-42af-b2ff-b5d5705d9480 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:33.192300 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 828/1680] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:48:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:48:33.196933 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70eaef-2f08-4584-b690-3221bf51832d None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:33.238380 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70eaef-2f08-4584-b690-3221bf51832d None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:33.252010 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f336c377-1231-4b65-90b1-1a9a15eddb6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:33.253287 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336c377-1231-4b65-90b1-1a9a15eddb6f admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:33.253965 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336c377-1231-4b65-90b1-1a9a15eddb6f admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:33.254583 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336c377-1231-4b65-90b1-1a9a15eddb6f admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:33.255817 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f336c377-1231-4b65-90b1-1a9a15eddb6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:33.258556 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f336c377-1231-4b65-90b1-1a9a15eddb6f admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:33.392246 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 853/1681] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:48:33 2022] POST /identity/v3/projects => generated 373 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:48:33.397546 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecbe2f9-7add-42af-b2ff-b5d5705d9480 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:33.433466 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecbe2f9-7add-42af-b2ff-b5d5705d9480 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:33.658751 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f336c377-1231-4b65-90b1-1a9a15eddb6f admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:33.661371 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-f336c377-1231-4b65-90b1-1a9a15eddb6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:33.676043 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7590d-b5f2-4483-bba1-f3bc1ea2a265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:33.677638 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7590d-b5f2-4483-bba1-f3bc1ea2a265 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:33.678084 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7590d-b5f2-4483-bba1-f3bc1ea2a265 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:33.678507 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7590d-b5f2-4483-bba1-f3bc1ea2a265 admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:33.679308 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e7590d-b5f2-4483-bba1-f3bc1ea2a265 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:33.683163 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e7590d-b5f2-4483-bba1-f3bc1ea2a265 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:33.855006 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f336c377-1231-4b65-90b1-1a9a15eddb6f admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:33.865473 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 829/1682] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:48:33 2022] POST /identity/v3/users => generated 287 bytes in 465 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:48:33.866609 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7590d-b5f2-4483-bba1-f3bc1ea2a265 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:33.867520 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7500a52e-10d6-4531-ad14-886a2a28364c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:33.868176 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500a52e-10d6-4531-ad14-886a2a28364c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:33.868463 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500a52e-10d6-4531-ad14-886a2a28364c admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:33.868744 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500a52e-10d6-4531-ad14-886a2a28364c admin admin] PATH_INFO: `/v3/users/3a5e59d1799a4b0ba6dda59d77e91256` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:33.872445 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7500a52e-10d6-4531-ad14-886a2a28364c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a5e59d1799a4b0ba6dda59d77e91256)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:33.875674 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7500a52e-10d6-4531-ad14-886a2a28364c admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:33.886701 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7590d-b5f2-4483-bba1-f3bc1ea2a265 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:33.901462 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f364a-9751-4a73-b7a0-35495ef36b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:33.902096 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f364a-9751-4a73-b7a0-35495ef36b35 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:33.902380 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f364a-9751-4a73-b7a0-35495ef36b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:33.902657 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f364a-9751-4a73-b7a0-35495ef36b35 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:33.903311 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f364a-9751-4a73-b7a0-35495ef36b35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:33.904978 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f364a-9751-4a73-b7a0-35495ef36b35 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:33.933075 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 830/1683] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:48:33 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:48:33.936629 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f364a-9751-4a73-b7a0-35495ef36b35 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:33.963182 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f364a-9751-4a73-b7a0-35495ef36b35 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:33.971403 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f69fa9-4f99-4c70-b226-a96d6539c1ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:33.972293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f69fa9-4f99-4c70-b226-a96d6539c1ee admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:33.972742 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f69fa9-4f99-4c70-b226-a96d6539c1ee admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:33.973161 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f69fa9-4f99-4c70-b226-a96d6539c1ee admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:33.974022 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f69fa9-4f99-4c70-b226-a96d6539c1ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:33.975816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f69fa9-4f99-4c70-b226-a96d6539c1ee admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:33.995104 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 831/1684] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:48:33 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:48:34.004318 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f69fa9-4f99-4c70-b226-a96d6539c1ee admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:34.022034 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f69fa9-4f99-4c70-b226-a96d6539c1ee admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:34.037492 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2ddc54-f617-492f-bca1-056ff22ba946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:34.039602 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2ddc54-f617-492f-bca1-056ff22ba946 admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:34.039973 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2ddc54-f617-492f-bca1-056ff22ba946 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:34.041226 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2ddc54-f617-492f-bca1-056ff22ba946 admin admin] PATH_INFO: `/v3/projects/c3e5330d062b461a911237be416225db/users/199f6c2d23ef43668af02a613bb05dd8/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:34.110955 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2ddc54-f617-492f-bca1-056ff22ba946 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3e5330d062b461a911237be416225db, user_id=199f6c2d23ef43668af02a613bb05dd8, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:34.126464 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2ddc54-f617-492f-bca1-056ff22ba946 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:34.245908 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-7500a52e-10d6-4531-ad14-886a2a28364c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a5e59d1799a4b0ba6dda59d77e91256'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:34.272809 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 832/1685] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:48:34 2022] PUT /identity/v3/projects/c3e5330d062b461a911237be416225db/users/199f6c2d23ef43668af02a613bb05dd8/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:48:34.284134 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c6f50-0985-4c08-b36c-0db364733241 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:34.284838 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c6f50-0985-4c08-b36c-0db364733241 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:34.286356 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c6f50-0985-4c08-b36c-0db364733241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:34.376303 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-7500a52e-10d6-4531-ad14-886a2a28364c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a5e59d1799a4b0ba6dda59d77e91256'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:34.418781 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-7500a52e-10d6-4531-ad14-886a2a28364c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a5e59d1799a4b0ba6dda59d77e91256'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:34.422608 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-a22c6f50-0985-4c08-b36c-0db364733241 None None] MFA Rules not processed for user `199f6c2d23ef43668af02a613bb05dd8`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:34.427240 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-a22c6f50-0985-4c08-b36c-0db364733241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:34.441011 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-7500a52e-10d6-4531-ad14-886a2a28364c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a5e59d1799a4b0ba6dda59d77e91256'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:34.444671 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-7500a52e-10d6-4531-ad14-886a2a28364c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a5e59d1799a4b0ba6dda59d77e91256'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:34.462214 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 854/1686] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:48:33 2022] DELETE /identity/v3/users/3a5e59d1799a4b0ba6dda59d77e91256 => generated 0 bytes in 806 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:48:34.466177 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0510b11-61e1-4e93-baf3-0db2bad94eeb None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:34.467544 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0510b11-61e1-4e93-baf3-0db2bad94eeb None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:34.468577 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0510b11-61e1-4e93-baf3-0db2bad94eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:34.496946 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-f0510b11-61e1-4e93-baf3-0db2bad94eeb None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:34.498892 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-f0510b11-61e1-4e93-baf3-0db2bad94eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:34.607230 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:34.739175 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 833/1687] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 456 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:34.749945 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c6f50-0985-4c08-b36c-0db364733241 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:34.751915 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-a22c6f50-0985-4c08-b36c-0db364733241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:34.756573 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:34.805757 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 855/1688] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:34.810690 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae9e4f-352c-4b49-a5dd-1d6fcc64fe2c None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:34.813426 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae9e4f-352c-4b49-a5dd-1d6fcc64fe2c None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:34.813426 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae9e4f-352c-4b49-a5dd-1d6fcc64fe2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:34.865177 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-97ae9e4f-352c-4b49-a5dd-1d6fcc64fe2c None None] MFA Rules not processed for user `53f9f8eb7a20431ca0d679af45946bce`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:34.869056 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-97ae9e4f-352c-4b49-a5dd-1d6fcc64fe2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:34.991158 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c6f50-0985-4c08-b36c-0db364733241 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:35.020900 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0850e6e9-6bac-47d2-9c1b-cee738faf243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:35.021552 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0850e6e9-6bac-47d2-9c1b-cee738faf243 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:35.021801 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0850e6e9-6bac-47d2-9c1b-cee738faf243 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:35.022252 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0850e6e9-6bac-47d2-9c1b-cee738faf243 admin admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:35.023069 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0850e6e9-6bac-47d2-9c1b-cee738faf243 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:35.028234 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0850e6e9-6bac-47d2-9c1b-cee738faf243 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:35.063774 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 856/1689] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:34 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:35.066526 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae9e4f-352c-4b49-a5dd-1d6fcc64fe2c None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:35.070952 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-97ae9e4f-352c-4b49-a5dd-1d6fcc64fe2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:35.194663 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae9e4f-352c-4b49-a5dd-1d6fcc64fe2c None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:35.198827 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e03e54-1d53-4eda-b138-6c338856870b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:35.200305 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e03e54-1d53-4eda-b138-6c338856870b service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:35.200640 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e03e54-1d53-4eda-b138-6c338856870b service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:35.200954 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e03e54-1d53-4eda-b138-6c338856870b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:35.222782 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 834/1690] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:48:34 2022] POST /identity/v3/projects => generated 371 bytes in 470 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:48:35.223940 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0850e6e9-6bac-47d2-9c1b-cee738faf243 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:35.242718 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e03e54-1d53-4eda-b138-6c338856870b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:35.249846 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e03e54-1d53-4eda-b138-6c338856870b service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:35.274663 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 857/1691] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:48:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:48:35.404903 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0850e6e9-6bac-47d2-9c1b-cee738faf243 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:35.408829 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53179c-8184-4208-87aa-15275e4563b8 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:35.409125 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53179c-8184-4208-87aa-15275e4563b8 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:35.413322 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53179c-8184-4208-87aa-15275e4563b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:35.418797 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce62eaa-ae70-4eb4-be7e-c64380bc1267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:35.419428 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce62eaa-ae70-4eb4-be7e-c64380bc1267 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:35.419712 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce62eaa-ae70-4eb4-be7e-c64380bc1267 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:35.419990 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce62eaa-ae70-4eb4-be7e-c64380bc1267 admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:35.420630 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce62eaa-ae70-4eb4-be7e-c64380bc1267 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:35.422879 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce62eaa-ae70-4eb4-be7e-c64380bc1267 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:35.450726 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-bb53179c-8184-4208-87aa-15275e4563b8 None None] MFA Rules not processed for user `bcd83a222eb04368a409db06389f1b7d`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:35.452876 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-bb53179c-8184-4208-87aa-15275e4563b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:35.558396 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 835/1692] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:48:35 2022] POST /identity/v3/users => generated 292 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:48:35.561005 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0998935-79f6-496f-bb7c-1352e8a3126c None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:35.562135 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0998935-79f6-496f-bb7c-1352e8a3126c None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:35.562686 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0998935-79f6-496f-bb7c-1352e8a3126c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:35.571467 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 858/1693] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:35.572570 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb53179c-8184-4208-87aa-15275e4563b8 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:35.592101 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb53179c-8184-4208-87aa-15275e4563b8 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:35.601782 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4bde62-9c8a-445c-a0ee-ca26851453c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:35.604649 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4bde62-9c8a-445c-a0ee-ca26851453c0 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:35.604968 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4bde62-9c8a-445c-a0ee-ca26851453c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:35.605253 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4bde62-9c8a-445c-a0ee-ca26851453c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:35.605925 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4bde62-9c8a-445c-a0ee-ca26851453c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:35.609597 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4bde62-9c8a-445c-a0ee-ca26851453c0 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:35.639122 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 859/1694] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:48:35 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:48:35.640854 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4bde62-9c8a-445c-a0ee-ca26851453c0 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:35.657007 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4bde62-9c8a-445c-a0ee-ca26851453c0 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:35.667184 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603802e0-121c-4519-b232-75e86dc6c99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:35.668155 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603802e0-121c-4519-b232-75e86dc6c99e admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:35.668660 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603802e0-121c-4519-b232-75e86dc6c99e admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:35.669128 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603802e0-121c-4519-b232-75e86dc6c99e admin admin] PATH_INFO: `/v3/projects/77a02e4255524676bcbc5e4ef5a018b8/users/adf05ffc255d435abeb2cfe4d2c95748/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:35.687682 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-a0998935-79f6-496f-bb7c-1352e8a3126c None None] MFA Rules not processed for user `f77dfe78b1a8405084c06d9acee96511`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:35.690137 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-a0998935-79f6-496f-bb7c-1352e8a3126c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:35.789415 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603802e0-121c-4519-b232-75e86dc6c99e admin admin] RBAC: Authorizing `identity:create_grant(project_id=77a02e4255524676bcbc5e4ef5a018b8, user_id=adf05ffc255d435abeb2cfe4d2c95748, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:35.798202 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603802e0-121c-4519-b232-75e86dc6c99e admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:35.823047 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:35.892881 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 860/1695] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:48:35 2022] PUT /identity/v3/projects/77a02e4255524676bcbc5e4ef5a018b8/users/adf05ffc255d435abeb2cfe4d2c95748/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:48:35.898603 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 836/1696] 10.222.0.52 () {58 vars in 1009 bytes} [Mon Aug 29 22:48:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:35.899274 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603802e0-121c-4519-b232-75e86dc6c99e admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:36.122589 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603802e0-121c-4519-b232-75e86dc6c99e admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:36.145360 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853eea14-6f28-43a1-b44f-30f7b6b84cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:36.146235 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853eea14-6f28-43a1-b44f-30f7b6b84cd4 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:36.146670 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853eea14-6f28-43a1-b44f-30f7b6b84cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:36.147081 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853eea14-6f28-43a1-b44f-30f7b6b84cd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:36.147901 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853eea14-6f28-43a1-b44f-30f7b6b84cd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:36.149815 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853eea14-6f28-43a1-b44f-30f7b6b84cd4 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:36.185514 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 861/1697] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:48:35 2022] GET /identity/v3/roles => generated 1420 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:48:36.188645 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0998935-79f6-496f-bb7c-1352e8a3126c None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:36.209642 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0998935-79f6-496f-bb7c-1352e8a3126c None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:36.256701 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e758e580-7a23-4ed2-b0d1-c1ea45bb5ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:36.259616 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758e580-7a23-4ed2-b0d1-c1ea45bb5ed4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:36.260659 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758e580-7a23-4ed2-b0d1-c1ea45bb5ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:36.261595 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758e580-7a23-4ed2-b0d1-c1ea45bb5ed4 admin admin] PATH_INFO: `/v3/projects/77a02e4255524676bcbc5e4ef5a018b8/users/adf05ffc255d435abeb2cfe4d2c95748/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:36.266943 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e758e580-7a23-4ed2-b0d1-c1ea45bb5ed4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77a02e4255524676bcbc5e4ef5a018b8, user_id=adf05ffc255d435abeb2cfe4d2c95748, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:36.272292 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e758e580-7a23-4ed2-b0d1-c1ea45bb5ed4 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:36.353302 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 837/1698] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:48:36 2022] PUT /identity/v3/projects/77a02e4255524676bcbc5e4ef5a018b8/users/adf05ffc255d435abeb2cfe4d2c95748/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:48:36.368836 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd26c6a8-224c-43ed-bb46-b5f5bdaf2670 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:36.373082 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd26c6a8-224c-43ed-bb46-b5f5bdaf2670 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:36.373082 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd26c6a8-224c-43ed-bb46-b5f5bdaf2670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:36.436198 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e758e580-7a23-4ed2-b0d1-c1ea45bb5ed4 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:36.438711 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-e758e580-7a23-4ed2-b0d1-c1ea45bb5ed4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:36.447854 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-dd26c6a8-224c-43ed-bb46-b5f5bdaf2670 None None] MFA Rules not processed for user `adf05ffc255d435abeb2cfe4d2c95748`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:36.455040 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-dd26c6a8-224c-43ed-bb46-b5f5bdaf2670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:36.687238 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 862/1699] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:36 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:36.688314 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd26c6a8-224c-43ed-bb46-b5f5bdaf2670 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:36.690965 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-dd26c6a8-224c-43ed-bb46-b5f5bdaf2670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:36.702119 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e758e580-7a23-4ed2-b0d1-c1ea45bb5ed4 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:36.729661 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b5cda-0b63-445e-bd9d-23177dccb650 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57c175ac90e545358746176972ff39a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:36.732052 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b5cda-0b63-445e-bd9d-23177dccb650 service swift] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:36.733067 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b5cda-0b63-445e-bd9d-23177dccb650 service swift] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:36.734100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b5cda-0b63-445e-bd9d-23177dccb650 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:36.787101 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:36.906468 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd26c6a8-224c-43ed-bb46-b5f5bdaf2670 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:36.921708 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea50b205-8411-4075-b3c0-e28e88a04edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:36.922628 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146b5cda-0b63-445e-bd9d-23177dccb650 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:36.922720 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50b205-8411-4075-b3c0-e28e88a04edc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:36.922720 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50b205-8411-4075-b3c0-e28e88a04edc admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:36.922720 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50b205-8411-4075-b3c0-e28e88a04edc admin admin] PATH_INFO: `/v3/users/75a78417cb914a10ab735d13b49aecff` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:36.922967 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146b5cda-0b63-445e-bd9d-23177dccb650 service swift] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:36.924120 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea50b205-8411-4075-b3c0-e28e88a04edc admin admin] RBAC: Authorizing `identity:delete_user(user_id=75a78417cb914a10ab735d13b49aecff)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:36.930700 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea50b205-8411-4075-b3c0-e28e88a04edc admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:36.943948 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 838/1700] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Aug 29 22:48:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 22:48:36.949187 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b5cda-0b63-445e-bd9d-23177dccb650 service swift] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:36.952022 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-146b5cda-0b63-445e-bd9d-23177dccb650 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:37.082386 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:37.199432 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b5cda-0b63-445e-bd9d-23177dccb650 service swift] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:37.211961 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e04e18-134e-40e4-8896-c5697c995a82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:37.215126 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e04e18-134e-40e4-8896-c5697c995a82 service cinder] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:37.215733 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e04e18-134e-40e4-8896-c5697c995a82 service cinder] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:37.216384 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e04e18-134e-40e4-8896-c5697c995a82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:37.251635 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e04e18-134e-40e4-8896-c5697c995a82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:37.274561 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e04e18-134e-40e4-8896-c5697c995a82 service cinder] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:37.319940 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 839/1701] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:48:36 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:48:37.319940 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d61c3-d36c-4e66-826f-931690ef2931 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:37.319940 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d61c3-d36c-4e66-826f-931690ef2931 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:37.319940 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d61c3-d36c-4e66-826f-931690ef2931 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:37.319940 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 840/1702] 10.222.0.52 () {54 vars in 912 bytes} [Mon Aug 29 22:48:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:48:37.333569 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c2fdb-7f7b-4c0c-a5e0-d3c17e4f4a38 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:37.334111 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c2fdb-7f7b-4c0c-a5e0-d3c17e4f4a38 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:37.335091 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c2fdb-7f7b-4c0c-a5e0-d3c17e4f4a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:37.393431 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-278c2fdb-7f7b-4c0c-a5e0-d3c17e4f4a38 None None] MFA Rules not processed for user `0fe5e8c4cb324ffb9d5b80f18d123727`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:37.395768 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-278c2fdb-7f7b-4c0c-a5e0-d3c17e4f4a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:37.410271 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ea50b205-8411-4075-b3c0-e28e88a04edc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75a78417cb914a10ab735d13b49aecff'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:37.606441 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 841/1703] 10.222.0.52 () {58 vars in 1009 bytes} [Mon Aug 29 22:48:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:37.620765 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ea50b205-8411-4075-b3c0-e28e88a04edc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75a78417cb914a10ab735d13b49aecff'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:37.671642 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c2fdb-7f7b-4c0c-a5e0-d3c17e4f4a38 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:37.678593 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ea50b205-8411-4075-b3c0-e28e88a04edc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75a78417cb914a10ab735d13b49aecff'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:37.715459 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c2fdb-7f7b-4c0c-a5e0-d3c17e4f4a38 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:37.715894 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ea50b205-8411-4075-b3c0-e28e88a04edc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75a78417cb914a10ab735d13b49aecff'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:37.718096 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ea50b205-8411-4075-b3c0-e28e88a04edc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75a78417cb914a10ab735d13b49aecff'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:37.735369 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e6eee8-9d50-46c3-ba8f-07adac3141e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:37.736832 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6eee8-9d50-46c3-ba8f-07adac3141e7 service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:37.739583 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6eee8-9d50-46c3-ba8f-07adac3141e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:37.743272 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6eee8-9d50-46c3-ba8f-07adac3141e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:37.748231 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-79e6eee8-9d50-46c3-ba8f-07adac3141e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:37.755373 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 863/1704] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:48:36 2022] DELETE /identity/v3/users/75a78417cb914a10ab735d13b49aecff => generated 0 bytes in 1067 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:48:37.762916 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d0a1a-7845-40b6-8e05-0102fce0e9be None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:37.765758 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d0a1a-7845-40b6-8e05-0102fce0e9be None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:37.766227 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d0a1a-7845-40b6-8e05-0102fce0e9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:37.788377 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e6eee8-9d50-46c3-ba8f-07adac3141e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:37.792329 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e6eee8-9d50-46c3-ba8f-07adac3141e7 service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:37.816563 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-a88d0a1a-7845-40b6-8e05-0102fce0e9be None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:37.830317 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-a88d0a1a-7845-40b6-8e05-0102fce0e9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:37.838093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 842/1705] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:48:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:48:37.940477 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 864/1706] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:37.963913 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e6eee8-9d50-46c3-ba8f-07adac3141e7 service neutron] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:37.965498 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-79e6eee8-9d50-46c3-ba8f-07adac3141e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:38.000198 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e6eee8-9d50-46c3-ba8f-07adac3141e7 service neutron] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:38.200760 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c682a-0e46-44c0-a0aa-1a4dde1f798e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:38.204619 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c682a-0e46-44c0-a0aa-1a4dde1f798e service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:38.216500 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c682a-0e46-44c0-a0aa-1a4dde1f798e service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:38.219899 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c682a-0e46-44c0-a0aa-1a4dde1f798e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:38.275926 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3c682a-0e46-44c0-a0aa-1a4dde1f798e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:38.277570 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3c682a-0e46-44c0-a0aa-1a4dde1f798e service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:38.331542 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 843/1707] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:48:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:48:38.337420 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d0a1a-7845-40b6-8e05-0102fce0e9be None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:38.339915 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-a88d0a1a-7845-40b6-8e05-0102fce0e9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:38.364981 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d0a1a-7845-40b6-8e05-0102fce0e9be None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:38.376812 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a5fb9e-c5a8-4dd3-ba07-d00d370e5e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:38.378415 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5fb9e-c5a8-4dd3-ba07-d00d370e5e2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:38.382218 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5fb9e-c5a8-4dd3-ba07-d00d370e5e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:38.382830 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5fb9e-c5a8-4dd3-ba07-d00d370e5e2e admin admin] PATH_INFO: `/v3/projects/91c111b2b38c4938ac717066e64a7262` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:38.384592 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a5fb9e-c5a8-4dd3-ba07-d00d370e5e2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=91c111b2b38c4938ac717066e64a7262)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:38.394389 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a5fb9e-c5a8-4dd3-ba07-d00d370e5e2e admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:38.400464 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:38.586360 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-13a5fb9e-c5a8-4dd3-ba07-d00d370e5e2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91c111b2b38c4938ac717066e64a7262', 'request_id': 'req-13a5fb9e-c5a8-4dd3-ba07-d00d370e5e2e'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:38.602426 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-13a5fb9e-c5a8-4dd3-ba07-d00d370e5e2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91c111b2b38c4938ac717066e64a7262', 'request_id': 'req-13a5fb9e-c5a8-4dd3-ba07-d00d370e5e2e'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:38.620983 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-13a5fb9e-c5a8-4dd3-ba07-d00d370e5e2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91c111b2b38c4938ac717066e64a7262', 'request_id': 'req-13a5fb9e-c5a8-4dd3-ba07-d00d370e5e2e'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:38.765802 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-13a5fb9e-c5a8-4dd3-ba07-d00d370e5e2e admin admin] The token cache is being invalidate because project 91c111b2b38c4938ac717066e64a7262 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:48:38.769160 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-13a5fb9e-c5a8-4dd3-ba07-d00d370e5e2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:38.769160 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-13a5fb9e-c5a8-4dd3-ba07-d00d370e5e2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:38.770633 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 865/1708] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:48:38 2022] DELETE /identity/v3/projects/91c111b2b38c4938ac717066e64a7262 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:48:38.776222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aac661-4ef0-4a57-978a-756389e8f06e None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:38.776509 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aac661-4ef0-4a57-978a-756389e8f06e None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:38.776993 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aac661-4ef0-4a57-978a-756389e8f06e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:38.830375 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-40aac661-4ef0-4a57-978a-756389e8f06e None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:38.833564 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-40aac661-4ef0-4a57-978a-756389e8f06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:38.991626 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f01196-36e2-425e-bee2-d06d0b32a0a5 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:38.998035 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f01196-36e2-425e-bee2-d06d0b32a0a5 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:39.000071 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f01196-36e2-425e-bee2-d06d0b32a0a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:39.106790 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-88f01196-36e2-425e-bee2-d06d0b32a0a5 None None] MFA Rules not processed for user `38a89946e1fb4b43aed992078dba0b4f`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:39.114423 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-88f01196-36e2-425e-bee2-d06d0b32a0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:39.193389 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 844/1709] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:39.199314 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db618384-aba7-49df-85a0-a738eb6e3636 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:39.211349 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db618384-aba7-49df-85a0-a738eb6e3636 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:39.211949 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db618384-aba7-49df-85a0-a738eb6e3636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:39.316643 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-db618384-aba7-49df-85a0-a738eb6e3636 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:39.320347 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-db618384-aba7-49df-85a0-a738eb6e3636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:39.365238 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 845/1710] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:39.371812 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db618384-aba7-49df-85a0-a738eb6e3636 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:39.392984 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 866/1711] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:38 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 402 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:39.403991 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db618384-aba7-49df-85a0-a738eb6e3636 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:39.424691 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be7323-e283-494b-ba2c-e23dddf9f3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:39.425349 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be7323-e283-494b-ba2c-e23dddf9f3b2 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:39.425702 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be7323-e283-494b-ba2c-e23dddf9f3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:39.425978 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be7323-e283-494b-ba2c-e23dddf9f3b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:39.426635 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20be7323-e283-494b-ba2c-e23dddf9f3b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:48:39.426980 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20be7323-e283-494b-ba2c-e23dddf9f3b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:39.432281 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20be7323-e283-494b-ba2c-e23dddf9f3b2 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:39.434262 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f01196-36e2-425e-bee2-d06d0b32a0a5 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:39.437108 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-88f01196-36e2-425e-bee2-d06d0b32a0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:39.493349 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:39.494181 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f01196-36e2-425e-bee2-d06d0b32a0a5 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:39.505864 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 846/1712] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:48:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:48:39.509146 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be7323-e283-494b-ba2c-e23dddf9f3b2 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:39.517013 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556b284-d5e4-4694-b0a6-3f8664678e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:39.517013 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556b284-d5e4-4694-b0a6-3f8664678e47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:39.517013 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556b284-d5e4-4694-b0a6-3f8664678e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:39.517013 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556b284-d5e4-4694-b0a6-3f8664678e47 admin admin] PATH_INFO: `/v3/users/199f6c2d23ef43668af02a613bb05dd8` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:39.517593 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5556b284-d5e4-4694-b0a6-3f8664678e47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=199f6c2d23ef43668af02a613bb05dd8)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:39.527651 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5556b284-d5e4-4694-b0a6-3f8664678e47 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:39.552378 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be7323-e283-494b-ba2c-e23dddf9f3b2 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:39.573695 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf7e9d6-da3a-4695-b931-543f1ec34555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:39.574964 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf7e9d6-da3a-4695-b931-543f1ec34555 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:39.575404 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf7e9d6-da3a-4695-b931-543f1ec34555 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:39.575832 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf7e9d6-da3a-4695-b931-543f1ec34555 admin admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:39.576617 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf7e9d6-da3a-4695-b931-543f1ec34555 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:39.579502 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf7e9d6-da3a-4695-b931-543f1ec34555 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:39.581057 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:39.864995 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 847/1713] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:48:39 2022] POST /identity/v3/projects => generated 365 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:48:39.866304 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf7e9d6-da3a-4695-b931-543f1ec34555 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:39.870890 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-fbf7e9d6-da3a-4695-b931-543f1ec34555 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:40.007126 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-5556b284-d5e4-4694-b0a6-3f8664678e47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '199f6c2d23ef43668af02a613bb05dd8'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:40.128836 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-5556b284-d5e4-4694-b0a6-3f8664678e47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '199f6c2d23ef43668af02a613bb05dd8'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:40.200653 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-5556b284-d5e4-4694-b0a6-3f8664678e47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '199f6c2d23ef43668af02a613bb05dd8'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:40.237921 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-5556b284-d5e4-4694-b0a6-3f8664678e47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '199f6c2d23ef43668af02a613bb05dd8'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:40.241251 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-5556b284-d5e4-4694-b0a6-3f8664678e47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '199f6c2d23ef43668af02a613bb05dd8'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:40.263277 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 867/1714] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:48:39 2022] DELETE /identity/v3/users/199f6c2d23ef43668af02a613bb05dd8 => generated 0 bytes in 830 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:48:40.267214 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556b284-d5e4-4694-b0a6-3f8664678e47 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:40.268347 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf7e9d6-da3a-4695-b931-543f1ec34555 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:40.272182 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-5556b284-d5e4-4694-b0a6-3f8664678e47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:40.457716 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57857962-7144-44e0-8b93-542441f14d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:40.459669 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57857962-7144-44e0-8b93-542441f14d99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:40.460415 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57857962-7144-44e0-8b93-542441f14d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:40.462281 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57857962-7144-44e0-8b93-542441f14d99 admin admin] PATH_INFO: `/v3/users/4971a195508f400998673f7cf49d54ac` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:40.468464 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57857962-7144-44e0-8b93-542441f14d99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4971a195508f400998673f7cf49d54ac)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:40.474538 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57857962-7144-44e0-8b93-542441f14d99 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:40.483683 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556b284-d5e4-4694-b0a6-3f8664678e47 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:40.498449 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6abb28-fa41-467c-9b52-727cad13d364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:40.500542 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6abb28-fa41-467c-9b52-727cad13d364 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:40.501907 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6abb28-fa41-467c-9b52-727cad13d364 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:40.502923 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6abb28-fa41-467c-9b52-727cad13d364 admin admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:40.504355 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6abb28-fa41-467c-9b52-727cad13d364 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:40.509381 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6abb28-fa41-467c-9b52-727cad13d364 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:40.715970 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 868/1715] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:48:40 2022] POST /identity/v3/users => generated 283 bytes in 452 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:48:40.727710 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8ce23-a422-4121-a120-fbfee62c5f8a None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:40.728947 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8ce23-a422-4121-a120-fbfee62c5f8a None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:40.732093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8ce23-a422-4121-a120-fbfee62c5f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:40.819255 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-57857962-7144-44e0-8b93-542441f14d99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4971a195508f400998673f7cf49d54ac'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:40.836395 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-3ff8ce23-a422-4121-a120-fbfee62c5f8a None None] MFA Rules not processed for user `0fe5e8c4cb324ffb9d5b80f18d123727`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:40.841511 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-3ff8ce23-a422-4121-a120-fbfee62c5f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:40.872392 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-57857962-7144-44e0-8b93-542441f14d99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4971a195508f400998673f7cf49d54ac'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:40.906308 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-57857962-7144-44e0-8b93-542441f14d99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4971a195508f400998673f7cf49d54ac'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:41.009063 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:41.038993 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-57857962-7144-44e0-8b93-542441f14d99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4971a195508f400998673f7cf49d54ac'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:41.039844 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-57857962-7144-44e0-8b93-542441f14d99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4971a195508f400998673f7cf49d54ac'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:41.040163 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 869/1716] 10.222.0.71 () {58 vars in 1015 bytes} [Mon Aug 29 22:48:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:41.059704 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 848/1717] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:48:39 2022] DELETE /identity/v3/users/4971a195508f400998673f7cf49d54ac => generated 0 bytes in 1194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:48:41.069273 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57857962-7144-44e0-8b93-542441f14d99 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:41.070925 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a801ca-685a-45f9-bd6b-8bbe5c1db1e7 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:41.072994 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a801ca-685a-45f9-bd6b-8bbe5c1db1e7 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:41.073307 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a801ca-685a-45f9-bd6b-8bbe5c1db1e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:41.075259 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-57857962-7144-44e0-8b93-542441f14d99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:41.105011 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:41.136173 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-c9a801ca-685a-45f9-bd6b-8bbe5c1db1e7 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:41.143131 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-c9a801ca-685a-45f9-bd6b-8bbe5c1db1e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:41.330060 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57857962-7144-44e0-8b93-542441f14d99 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:41.330667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 870/1718] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:41.331017 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a801ca-685a-45f9-bd6b-8bbe5c1db1e7 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:41.333411 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-c9a801ca-685a-45f9-bd6b-8bbe5c1db1e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:41.343809 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f41efd-0ee0-4652-a5bc-6fcbce451c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:41.343809 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f41efd-0ee0-4652-a5bc-6fcbce451c88 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:41.344513 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f41efd-0ee0-4652-a5bc-6fcbce451c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:41.344894 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f41efd-0ee0-4652-a5bc-6fcbce451c88 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:41.345577 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f41efd-0ee0-4652-a5bc-6fcbce451c88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:41.347368 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f41efd-0ee0-4652-a5bc-6fcbce451c88 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:41.360094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a801ca-685a-45f9-bd6b-8bbe5c1db1e7 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:41.369618 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360fc88f-2a9c-47b0-9757-bbc176f24575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:41.370465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360fc88f-2a9c-47b0-9757-bbc176f24575 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:41.370905 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360fc88f-2a9c-47b0-9757-bbc176f24575 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:41.371334 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360fc88f-2a9c-47b0-9757-bbc176f24575 admin admin] PATH_INFO: `/v3/projects/d223b229017e4f0fb5bea686e62ad992` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:41.372654 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360fc88f-2a9c-47b0-9757-bbc176f24575 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d223b229017e4f0fb5bea686e62ad992)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:41.375534 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360fc88f-2a9c-47b0-9757-bbc176f24575 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:41.401458 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 849/1719] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:48:41 2022] GET /identity/v3/roles => generated 1420 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:48:41.403004 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6b8897-a802-4fee-bb30-83ad1f191bfa None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:41.403362 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6b8897-a802-4fee-bb30-83ad1f191bfa None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:41.403666 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6b8897-a802-4fee-bb30-83ad1f191bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:41.454940 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-af6b8897-a802-4fee-bb30-83ad1f191bfa None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:41.456186 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-af6b8897-a802-4fee-bb30-83ad1f191bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:41.473876 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 850/1720] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:41.473876 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6b8897-a802-4fee-bb30-83ad1f191bfa None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:41.477406 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-af6b8897-a802-4fee-bb30-83ad1f191bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:41.574552 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6b8897-a802-4fee-bb30-83ad1f191bfa None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:41.587160 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-360fc88f-2a9c-47b0-9757-bbc176f24575 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd223b229017e4f0fb5bea686e62ad992', 'request_id': 'req-360fc88f-2a9c-47b0-9757-bbc176f24575'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:41.599680 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-360fc88f-2a9c-47b0-9757-bbc176f24575 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd223b229017e4f0fb5bea686e62ad992', 'request_id': 'req-360fc88f-2a9c-47b0-9757-bbc176f24575'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:41.611632 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-360fc88f-2a9c-47b0-9757-bbc176f24575 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd223b229017e4f0fb5bea686e62ad992', 'request_id': 'req-360fc88f-2a9c-47b0-9757-bbc176f24575'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:41.713797 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b393e37-8e07-4008-a4a3-3725d388aa74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:41.714332 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-360fc88f-2a9c-47b0-9757-bbc176f24575 admin admin] The token cache is being invalidate because project d223b229017e4f0fb5bea686e62ad992 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:48:41.715162 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b393e37-8e07-4008-a4a3-3725d388aa74 service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:41.715563 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-360fc88f-2a9c-47b0-9757-bbc176f24575 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:41.717188 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b393e37-8e07-4008-a4a3-3725d388aa74 service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:41.717403 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b393e37-8e07-4008-a4a3-3725d388aa74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:41.717530 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-360fc88f-2a9c-47b0-9757-bbc176f24575 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:41.724521 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 871/1721] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:48:41 2022] DELETE /identity/v3/projects/d223b229017e4f0fb5bea686e62ad992 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:48:41.724521 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360fc88f-2a9c-47b0-9757-bbc176f24575 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:41.725183 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-1b393e37-8e07-4008-a4a3-3725d388aa74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:41.727144 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-360fc88f-2a9c-47b0-9757-bbc176f24575 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:41.751856 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b393e37-8e07-4008-a4a3-3725d388aa74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:41.753482 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b393e37-8e07-4008-a4a3-3725d388aa74 service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:41.783022 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 851/1722] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:48:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:48:41.784104 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b393e37-8e07-4008-a4a3-3725d388aa74 service neutron] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:41.791856 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-1b393e37-8e07-4008-a4a3-3725d388aa74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:41.800209 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360fc88f-2a9c-47b0-9757-bbc176f24575 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:41.820548 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b1b40-e130-4c96-860b-928cbb12d15b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:41.821419 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b1b40-e130-4c96-860b-928cbb12d15b service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:41.836863 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b1b40-e130-4c96-860b-928cbb12d15b service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:41.837300 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b1b40-e130-4c96-860b-928cbb12d15b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:41.843141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-c30b1b40-e130-4c96-860b-928cbb12d15b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:42.015257 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b1b40-e130-4c96-860b-928cbb12d15b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:42.017608 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b1b40-e130-4c96-860b-928cbb12d15b service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:42.029167 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 872/1723] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:48:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:48:42.030865 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b1b40-e130-4c96-860b-928cbb12d15b service neutron] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:42.032463 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-c30b1b40-e130-4c96-860b-928cbb12d15b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:42.033243 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b393e37-8e07-4008-a4a3-3725d388aa74 service neutron] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:42.051962 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a85afb2-5f9d-488f-afd7-c1452b998206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:42.054056 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85afb2-5f9d-488f-afd7-c1452b998206 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:42.057564 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85afb2-5f9d-488f-afd7-c1452b998206 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:42.058009 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85afb2-5f9d-488f-afd7-c1452b998206 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:42.058819 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a85afb2-5f9d-488f-afd7-c1452b998206 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:42.063462 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b1b40-e130-4c96-860b-928cbb12d15b service neutron] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:42.076245 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a85afb2-5f9d-488f-afd7-c1452b998206 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:42.076571 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f44f2a-f80a-4804-8598-ef40d03a111b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:42.078127 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f44f2a-f80a-4804-8598-ef40d03a111b service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:42.078604 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f44f2a-f80a-4804-8598-ef40d03a111b service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:42.079065 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f44f2a-f80a-4804-8598-ef40d03a111b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:42.081299 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-c7f44f2a-f80a-4804-8598-ef40d03a111b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:42.121438 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 852/1724] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:48:41 2022] GET /identity/v3/roles => generated 1420 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:48:42.123562 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f44f2a-f80a-4804-8598-ef40d03a111b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:42.125540 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a85afb2-5f9d-488f-afd7-c1452b998206 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:42.132397 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f44f2a-f80a-4804-8598-ef40d03a111b service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:42.152977 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-3a85afb2-5f9d-488f-afd7-c1452b998206 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:42.161593 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 873/1725] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:48:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:48:42.163409 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f44f2a-f80a-4804-8598-ef40d03a111b service neutron] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:42.187528 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f44f2a-f80a-4804-8598-ef40d03a111b service neutron] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:42.206519 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dc8704-4b5d-43b4-872a-f4fac2a0ded3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:42.207404 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc8704-4b5d-43b4-872a-f4fac2a0ded3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:42.207759 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc8704-4b5d-43b4-872a-f4fac2a0ded3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:42.208105 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc8704-4b5d-43b4-872a-f4fac2a0ded3 admin admin] PATH_INFO: `/v3/projects/144daa4319574a589302ec2383685566/users/2a5fc4b25bcc4c4298c82c7950033a9a/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:42.217380 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a85afb2-5f9d-488f-afd7-c1452b998206 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:42.245243 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf12d5f-69d8-4c6e-8c2c-91c92add1fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:42.246814 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dc8704-4b5d-43b4-872a-f4fac2a0ded3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=144daa4319574a589302ec2383685566, user_id=2a5fc4b25bcc4c4298c82c7950033a9a, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:42.249644 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dc8704-4b5d-43b4-872a-f4fac2a0ded3 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:42.252602 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf12d5f-69d8-4c6e-8c2c-91c92add1fa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:42.252602 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf12d5f-69d8-4c6e-8c2c-91c92add1fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:42.258632 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf12d5f-69d8-4c6e-8c2c-91c92add1fa8 admin admin] PATH_INFO: `/v3/users/2528fff264524ad9b4155c5812c42e80` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:42.260120 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf12d5f-69d8-4c6e-8c2c-91c92add1fa8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2528fff264524ad9b4155c5812c42e80)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:42.263724 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf12d5f-69d8-4c6e-8c2c-91c92add1fa8 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:42.355636 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 874/1726] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:48:42 2022] PUT /identity/v3/projects/144daa4319574a589302ec2383685566/users/2a5fc4b25bcc4c4298c82c7950033a9a/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:48:42.369032 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb46e7-530f-497a-a037-6160e8f5262a None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:42.369387 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb46e7-530f-497a-a037-6160e8f5262a None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:42.369776 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb46e7-530f-497a-a037-6160e8f5262a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:42.445717 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-1dfb46e7-530f-497a-a037-6160e8f5262a None None] MFA Rules not processed for user `2a5fc4b25bcc4c4298c82c7950033a9a`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:42.449275 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-1dfb46e7-530f-497a-a037-6160e8f5262a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:42.493202 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-ddf12d5f-69d8-4c6e-8c2c-91c92add1fa8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2528fff264524ad9b4155c5812c42e80'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:42.513795 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-ddf12d5f-69d8-4c6e-8c2c-91c92add1fa8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2528fff264524ad9b4155c5812c42e80'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:42.527490 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-ddf12d5f-69d8-4c6e-8c2c-91c92add1fa8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2528fff264524ad9b4155c5812c42e80'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:42.648500 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-ddf12d5f-69d8-4c6e-8c2c-91c92add1fa8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2528fff264524ad9b4155c5812c42e80'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:42.649837 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-ddf12d5f-69d8-4c6e-8c2c-91c92add1fa8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2528fff264524ad9b4155c5812c42e80'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:42.659812 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 853/1727] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:48:42 2022] DELETE /identity/v3/users/2528fff264524ad9b4155c5812c42e80 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:48:42.675903 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 875/1728] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:42.696809 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf12d5f-69d8-4c6e-8c2c-91c92add1fa8 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:42.701610 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-ddf12d5f-69d8-4c6e-8c2c-91c92add1fa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:42.915120 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf12d5f-69d8-4c6e-8c2c-91c92add1fa8 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:42.920887 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8062b-51ea-4150-8bef-79bf2e0ee13d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:42.922072 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8062b-51ea-4150-8bef-79bf2e0ee13d service cinder] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:42.922605 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8062b-51ea-4150-8bef-79bf2e0ee13d service cinder] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:42.922976 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8062b-51ea-4150-8bef-79bf2e0ee13d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:42.925511 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-58f8062b-51ea-4150-8bef-79bf2e0ee13d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:42.929464 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:43.039121 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f8062b-51ea-4150-8bef-79bf2e0ee13d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:43.044925 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f8062b-51ea-4150-8bef-79bf2e0ee13d service cinder] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:43.086689 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 854/1729] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:48:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:48:43.380542 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:44.769111 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfb46e7-530f-497a-a037-6160e8f5262a None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:44.770842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8062b-51ea-4150-8bef-79bf2e0ee13d service cinder] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:44.771840 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-58f8062b-51ea-4150-8bef-79bf2e0ee13d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:44.911689 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:45.038819 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfb46e7-530f-497a-a037-6160e8f5262a None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:45.181579 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8062b-51ea-4150-8bef-79bf2e0ee13d service cinder] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:45.195258 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa85554-6d70-4ce7-b646-672b16c69cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:45.196964 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa85554-6d70-4ce7-b646-672b16c69cf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:45.200397 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa85554-6d70-4ce7-b646-672b16c69cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:45.200397 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa85554-6d70-4ce7-b646-672b16c69cf0 admin admin] PATH_INFO: `/v3/projects/68482ab19f6a4f068555b0762ba4ffce` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:45.204925 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa85554-6d70-4ce7-b646-672b16c69cf0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68482ab19f6a4f068555b0762ba4ffce)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:45.208133 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb816d6-5a3e-4396-9421-24e8cac5a6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:45.208807 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa85554-6d70-4ce7-b646-672b16c69cf0 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:45.209114 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb816d6-5a3e-4396-9421-24e8cac5a6fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:45.209518 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb816d6-5a3e-4396-9421-24e8cac5a6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:45.209930 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb816d6-5a3e-4396-9421-24e8cac5a6fb admin admin] PATH_INFO: `/v3/projects/7a82cd47e2cf488b9a81dc73eada77ef` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:45.211450 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb816d6-5a3e-4396-9421-24e8cac5a6fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a82cd47e2cf488b9a81dc73eada77ef)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:45.214032 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb816d6-5a3e-4396-9421-24e8cac5a6fb admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:45.247235 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:45.419930 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-8aa85554-6d70-4ce7-b646-672b16c69cf0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68482ab19f6a4f068555b0762ba4ffce', 'request_id': 'req-8aa85554-6d70-4ce7-b646-672b16c69cf0'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:45.647660 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-4bb816d6-5a3e-4396-9421-24e8cac5a6fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a82cd47e2cf488b9a81dc73eada77ef', 'request_id': 'req-4bb816d6-5a3e-4396-9421-24e8cac5a6fb'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:45.675005 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-8aa85554-6d70-4ce7-b646-672b16c69cf0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68482ab19f6a4f068555b0762ba4ffce', 'request_id': 'req-8aa85554-6d70-4ce7-b646-672b16c69cf0'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:45.683892 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-8aa85554-6d70-4ce7-b646-672b16c69cf0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68482ab19f6a4f068555b0762ba4ffce', 'request_id': 'req-8aa85554-6d70-4ce7-b646-672b16c69cf0'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:45.691141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-4bb816d6-5a3e-4396-9421-24e8cac5a6fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a82cd47e2cf488b9a81dc73eada77ef', 'request_id': 'req-4bb816d6-5a3e-4396-9421-24e8cac5a6fb'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:45.710659 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-8aa85554-6d70-4ce7-b646-672b16c69cf0 admin admin] The token cache is being invalidate because project 68482ab19f6a4f068555b0762ba4ffce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:48:45.710983 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-8aa85554-6d70-4ce7-b646-672b16c69cf0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:45.711730 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-8aa85554-6d70-4ce7-b646-672b16c69cf0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:45.717892 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 855/1730] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:48:44 2022] DELETE /identity/v3/projects/68482ab19f6a4f068555b0762ba4ffce => generated 0 bytes in 944 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:48:45.719982 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-4bb816d6-5a3e-4396-9421-24e8cac5a6fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a82cd47e2cf488b9a81dc73eada77ef', 'request_id': 'req-4bb816d6-5a3e-4396-9421-24e8cac5a6fb'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:45.720839 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3f6b9-1fa2-4949-a559-fc1546e04906 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:45.721260 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3f6b9-1fa2-4949-a559-fc1546e04906 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:45.721517 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3f6b9-1fa2-4949-a559-fc1546e04906 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:45.728695 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 856/1731] 10.222.0.52 () {54 vars in 921 bytes} [Mon Aug 29 22:48:45 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:48:45.738362 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3f6b9-1fa2-4949-a559-fc1546e04906 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:45.741660 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-abf3f6b9-1fa2-4949-a559-fc1546e04906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:45.825465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-4bb816d6-5a3e-4396-9421-24e8cac5a6fb admin admin] The token cache is being invalidate because project 7a82cd47e2cf488b9a81dc73eada77ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:48:45.826140 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-4bb816d6-5a3e-4396-9421-24e8cac5a6fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:45.830466 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-4bb816d6-5a3e-4396-9421-24e8cac5a6fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:45.832364 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 876/1732] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:48:44 2022] DELETE /identity/v3/projects/7a82cd47e2cf488b9a81dc73eada77ef => generated 0 bytes in 1064 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:48:45.833630 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb816d6-5a3e-4396-9421-24e8cac5a6fb admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:45.837850 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-4bb816d6-5a3e-4396-9421-24e8cac5a6fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:45.949230 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3f6b9-1fa2-4949-a559-fc1546e04906 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:45.968360 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874aa62c-23aa-4fd6-94af-39730a338bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:45.971217 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874aa62c-23aa-4fd6-94af-39730a338bf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:45.973746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874aa62c-23aa-4fd6-94af-39730a338bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:45.974302 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874aa62c-23aa-4fd6-94af-39730a338bf1 admin admin] PATH_INFO: `/v3/projects/c3e5330d062b461a911237be416225db` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:45.978688 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874aa62c-23aa-4fd6-94af-39730a338bf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3e5330d062b461a911237be416225db)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:45.986287 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874aa62c-23aa-4fd6-94af-39730a338bf1 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:46.123590 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb816d6-5a3e-4396-9421-24e8cac5a6fb admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:46.149293 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a7f95-742c-4303-add9-9c54936a7bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:46.149810 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a7f95-742c-4303-add9-9c54936a7bc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:46.150087 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a7f95-742c-4303-add9-9c54936a7bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:46.150358 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a7f95-742c-4303-add9-9c54936a7bc9 admin admin] PATH_INFO: `/v3/users/11328985ad094ae6840f564834710f16` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:46.158489 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a7f95-742c-4303-add9-9c54936a7bc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11328985ad094ae6840f564834710f16)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:46.162819 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a7f95-742c-4303-add9-9c54936a7bc9 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:46.177654 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-874aa62c-23aa-4fd6-94af-39730a338bf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3e5330d062b461a911237be416225db', 'request_id': 'req-874aa62c-23aa-4fd6-94af-39730a338bf1'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:46.257501 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-874aa62c-23aa-4fd6-94af-39730a338bf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3e5330d062b461a911237be416225db', 'request_id': 'req-874aa62c-23aa-4fd6-94af-39730a338bf1'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:46.271354 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-874aa62c-23aa-4fd6-94af-39730a338bf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3e5330d062b461a911237be416225db', 'request_id': 'req-874aa62c-23aa-4fd6-94af-39730a338bf1'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:46.309438 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-874aa62c-23aa-4fd6-94af-39730a338bf1 admin admin] The token cache is being invalidate because project c3e5330d062b461a911237be416225db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:48:46.310288 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-874aa62c-23aa-4fd6-94af-39730a338bf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:46.313601 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-874aa62c-23aa-4fd6-94af-39730a338bf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:46.315917 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 857/1733] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:48:45 2022] DELETE /identity/v3/projects/c3e5330d062b461a911237be416225db => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:48:46.321801 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f957fa-889b-4989-a35e-5020586612b7 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:46.322813 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f957fa-889b-4989-a35e-5020586612b7 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:46.323168 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f957fa-889b-4989-a35e-5020586612b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:46.356098 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-b0f957fa-889b-4989-a35e-5020586612b7 None None] MFA Rules not processed for user `bdbf961e02934e88af3a72d6218f32fa`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:46.357554 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-b0f957fa-889b-4989-a35e-5020586612b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:46.420545 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d24a7f95-742c-4303-add9-9c54936a7bc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11328985ad094ae6840f564834710f16'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:46.599098 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 858/1734] 10.222.0.52 () {58 vars in 1018 bytes} [Mon Aug 29 22:48:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:46.602659 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265b19a-f6d8-4fec-884a-837177a5892a None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:46.605177 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265b19a-f6d8-4fec-884a-837177a5892a None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:46.605664 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265b19a-f6d8-4fec-884a-837177a5892a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:46.686832 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-a265b19a-f6d8-4fec-884a-837177a5892a None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:46.695671 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-a265b19a-f6d8-4fec-884a-837177a5892a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:46.717446 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d24a7f95-742c-4303-add9-9c54936a7bc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11328985ad094ae6840f564834710f16'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:46.796606 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d24a7f95-742c-4303-add9-9c54936a7bc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11328985ad094ae6840f564834710f16'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:46.796606 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d24a7f95-742c-4303-add9-9c54936a7bc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11328985ad094ae6840f564834710f16'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:46.796606 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d24a7f95-742c-4303-add9-9c54936a7bc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11328985ad094ae6840f564834710f16'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:46.845958 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 877/1735] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:48:45 2022] DELETE /identity/v3/users/11328985ad094ae6840f564834710f16 => generated 0 bytes in 1013 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:48:46.848965 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a7f95-742c-4303-add9-9c54936a7bc9 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:46.852366 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-d24a7f95-742c-4303-add9-9c54936a7bc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:46.892855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 859/1736] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:46.907659 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35579933-e35a-44b6-ada5-df59defa160b None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:46.908249 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35579933-e35a-44b6-ada5-df59defa160b None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:46.908852 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35579933-e35a-44b6-ada5-df59defa160b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:46.975166 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-35579933-e35a-44b6-ada5-df59defa160b None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:46.977204 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-35579933-e35a-44b6-ada5-df59defa160b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:47.120454 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a7f95-742c-4303-add9-9c54936a7bc9 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:47.127362 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc4047f-44db-4a7e-bb92-503a940a3db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:47.127951 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4047f-44db-4a7e-bb92-503a940a3db7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:47.128241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4047f-44db-4a7e-bb92-503a940a3db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:47.128580 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4047f-44db-4a7e-bb92-503a940a3db7 admin admin] PATH_INFO: `/v3/users/adf05ffc255d435abeb2cfe4d2c95748` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:47.129753 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc4047f-44db-4a7e-bb92-503a940a3db7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=adf05ffc255d435abeb2cfe4d2c95748)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:47.132800 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc4047f-44db-4a7e-bb92-503a940a3db7 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:47.152093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 860/1737] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:48:47.157534 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35579933-e35a-44b6-ada5-df59defa160b None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:47.175518 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35579933-e35a-44b6-ada5-df59defa160b None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:47.189615 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2760e6-e861-4a38-b732-b08d105e4c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:47.190368 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2760e6-e861-4a38-b732-b08d105e4c98 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:47.190652 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2760e6-e861-4a38-b732-b08d105e4c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:47.190928 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2760e6-e861-4a38-b732-b08d105e4c98 admin admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:47.191593 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2760e6-e861-4a38-b732-b08d105e4c98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:48:47.191897 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2760e6-e861-4a38-b732-b08d105e4c98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:47.193558 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2760e6-e861-4a38-b732-b08d105e4c98 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:47.214819 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 861/1738] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:48:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:48:47.214819 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2760e6-e861-4a38-b732-b08d105e4c98 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:47.216602 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-bd2760e6-e861-4a38-b732-b08d105e4c98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:47.462420 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2760e6-e861-4a38-b732-b08d105e4c98 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:47.488367 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951fa92-90f3-4bda-8fd8-e5caeee7deed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:47.500267 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951fa92-90f3-4bda-8fd8-e5caeee7deed service nova] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:47.500267 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951fa92-90f3-4bda-8fd8-e5caeee7deed service nova] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:47.500267 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951fa92-90f3-4bda-8fd8-e5caeee7deed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:47.500533 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-cfc4047f-44db-4a7e-bb92-503a940a3db7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adf05ffc255d435abeb2cfe4d2c95748'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:47.500613 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-3951fa92-90f3-4bda-8fd8-e5caeee7deed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:47.534834 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3951fa92-90f3-4bda-8fd8-e5caeee7deed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:47.536672 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3951fa92-90f3-4bda-8fd8-e5caeee7deed service nova] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:47.667336 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 862/1739] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:48:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:48:47.667336 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951fa92-90f3-4bda-8fd8-e5caeee7deed service nova] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:47.667336 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951fa92-90f3-4bda-8fd8-e5caeee7deed service nova] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:47.667336 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbf842e-5a38-4834-926f-66c7246a8e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:47.668686 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf842e-5a38-4834-926f-66c7246a8e41 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:47.668686 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf842e-5a38-4834-926f-66c7246a8e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:47.668686 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf842e-5a38-4834-926f-66c7246a8e41 admin admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:47.668686 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbf842e-5a38-4834-926f-66c7246a8e41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:47.668686 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbf842e-5a38-4834-926f-66c7246a8e41 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:47.781047 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:47.843500 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-cfc4047f-44db-4a7e-bb92-503a940a3db7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adf05ffc255d435abeb2cfe4d2c95748'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:47.866421 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 863/1740] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:48:47 2022] POST /identity/v3/projects => generated 357 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:48:47.871693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbf842e-5a38-4834-926f-66c7246a8e41 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:47.915530 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-cfc4047f-44db-4a7e-bb92-503a940a3db7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adf05ffc255d435abeb2cfe4d2c95748'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:47.941688 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-cfc4047f-44db-4a7e-bb92-503a940a3db7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adf05ffc255d435abeb2cfe4d2c95748'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:47.943853 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-cfc4047f-44db-4a7e-bb92-503a940a3db7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adf05ffc255d435abeb2cfe4d2c95748'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:47.970550 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 878/1741] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:48:46 2022] DELETE /identity/v3/users/adf05ffc255d435abeb2cfe4d2c95748 => generated 0 bytes in 1121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:48:48.105423 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbf842e-5a38-4834-926f-66c7246a8e41 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:48.268598 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd3de7-ba4c-4aba-9066-d57811f163bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:48.272031 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd3de7-ba4c-4aba-9066-d57811f163bd admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:48.273675 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd3de7-ba4c-4aba-9066-d57811f163bd admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:48.273996 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd3de7-ba4c-4aba-9066-d57811f163bd admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:48.274662 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afd3de7-ba4c-4aba-9066-d57811f163bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:48.287062 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afd3de7-ba4c-4aba-9066-d57811f163bd admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:48.334691 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:48.406434 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 864/1742] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:48:47 2022] POST /identity/v3/users => generated 279 bytes in 533 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:48:48.408054 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc4047f-44db-4a7e-bb92-503a940a3db7 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:48.410157 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-cfc4047f-44db-4a7e-bb92-503a940a3db7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:48.450443 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc4047f-44db-4a7e-bb92-503a940a3db7 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:48.498133 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261e046d-0918-45c4-989f-1d86f6a12db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:48.499613 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e046d-0918-45c4-989f-1d86f6a12db5 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:48.502824 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e046d-0918-45c4-989f-1d86f6a12db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:48.508194 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e046d-0918-45c4-989f-1d86f6a12db5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:48.508194 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261e046d-0918-45c4-989f-1d86f6a12db5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:48.514353 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261e046d-0918-45c4-989f-1d86f6a12db5 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:48.578266 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd3de7-ba4c-4aba-9066-d57811f163bd admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:48.578371 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 879/1743] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:48:48.600346 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd3de7-ba4c-4aba-9066-d57811f163bd admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:48.630856 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c048445-dee3-4847-a55c-ad81c299913a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:48.635333 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c048445-dee3-4847-a55c-ad81c299913a admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:48.638983 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c048445-dee3-4847-a55c-ad81c299913a admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:48.646087 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c048445-dee3-4847-a55c-ad81c299913a admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:48.648375 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c048445-dee3-4847-a55c-ad81c299913a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:48.653886 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c048445-dee3-4847-a55c-ad81c299913a admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:48.717302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261e046d-0918-45c4-989f-1d86f6a12db5 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:48.721105 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 865/1744] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:48:48.756226 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261e046d-0918-45c4-989f-1d86f6a12db5 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:48.786648 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59210289-dff0-4b62-a82f-feb110251490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:48.799589 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59210289-dff0-4b62-a82f-feb110251490 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:48.800576 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59210289-dff0-4b62-a82f-feb110251490 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:48.801336 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59210289-dff0-4b62-a82f-feb110251490 admin admin] PATH_INFO: `/v3/projects/d1ba01bfd210473cb1eef75894b28b32/users/88497a4e7f5c4f73a6f1627a5c8e9d66/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:48.870111 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59210289-dff0-4b62-a82f-feb110251490 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1ba01bfd210473cb1eef75894b28b32, user_id=88497a4e7f5c4f73a6f1627a5c8e9d66, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:48.880684 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59210289-dff0-4b62-a82f-feb110251490 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:49.060605 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 880/1745] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:48:48 2022] PUT /identity/v3/projects/d1ba01bfd210473cb1eef75894b28b32/users/88497a4e7f5c4f73a6f1627a5c8e9d66/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:48:49.078239 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2aadf7-6fd1-43a2-bfa7-e200158d962e None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:49.078477 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2aadf7-6fd1-43a2-bfa7-e200158d962e None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:49.078803 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2aadf7-6fd1-43a2-bfa7-e200158d962e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:49.115338 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-ef2aadf7-6fd1-43a2-bfa7-e200158d962e None None] MFA Rules not processed for user `88497a4e7f5c4f73a6f1627a5c8e9d66`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:49.116550 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-ef2aadf7-6fd1-43a2-bfa7-e200158d962e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:49.317618 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 866/1746] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:49 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:49.320654 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59210289-dff0-4b62-a82f-feb110251490 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:49.434694 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59210289-dff0-4b62-a82f-feb110251490 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:49.457285 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d5ffc-7e6b-4347-9891-f74e0bf2cf8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:49.457285 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d5ffc-7e6b-4347-9891-f74e0bf2cf8c admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:49.457285 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d5ffc-7e6b-4347-9891-f74e0bf2cf8c admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:49.457285 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d5ffc-7e6b-4347-9891-f74e0bf2cf8c admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:49.457927 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114d5ffc-7e6b-4347-9891-f74e0bf2cf8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:49.457927 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114d5ffc-7e6b-4347-9891-f74e0bf2cf8c admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:49.639356 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 881/1747] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:48:49 2022] POST /identity/v3/projects => generated 357 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:48:49.659116 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2aadf7-6fd1-43a2-bfa7-e200158d962e None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:49.681611 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2aadf7-6fd1-43a2-bfa7-e200158d962e None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:49.880586 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fd2385-f7f8-4a3f-8638-8405b8f23e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:49.880586 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd2385-f7f8-4a3f-8638-8405b8f23e25 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:49.880586 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd2385-f7f8-4a3f-8638-8405b8f23e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:49.880586 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd2385-f7f8-4a3f-8638-8405b8f23e25 admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:49.882306 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fd2385-f7f8-4a3f-8638-8405b8f23e25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:49.882306 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fd2385-f7f8-4a3f-8638-8405b8f23e25 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:50.455723 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 867/1748] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:48:49 2022] POST /identity/v3/users => generated 285 bytes in 793 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:48:50.457405 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d5ffc-7e6b-4347-9891-f74e0bf2cf8c admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:50.478408 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fd2385-f7f8-4a3f-8638-8405b8f23e25 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:50.482166 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-06fd2385-f7f8-4a3f-8638-8405b8f23e25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:50.492977 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d5ffc-7e6b-4347-9891-f74e0bf2cf8c admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:50.543901 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e2ccb0-4d78-4341-a80c-d3d281b5581a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:50.546576 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e2ccb0-4d78-4341-a80c-d3d281b5581a admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:50.547830 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e2ccb0-4d78-4341-a80c-d3d281b5581a admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:50.549187 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e2ccb0-4d78-4341-a80c-d3d281b5581a admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:50.554165 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e2ccb0-4d78-4341-a80c-d3d281b5581a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:50.562345 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e2ccb0-4d78-4341-a80c-d3d281b5581a admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:50.569780 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fd2385-f7f8-4a3f-8638-8405b8f23e25 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:50.585369 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81ec421-1cbf-4d4f-b304-7719407468ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:50.586520 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ec421-1cbf-4d4f-b304-7719407468ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:50.586994 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ec421-1cbf-4d4f-b304-7719407468ff admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:50.587647 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ec421-1cbf-4d4f-b304-7719407468ff admin admin] PATH_INFO: `/v3/projects/00a715aebe514121866463e092d66fa5` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:50.589705 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81ec421-1cbf-4d4f-b304-7719407468ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=00a715aebe514121866463e092d66fa5)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:50.593266 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81ec421-1cbf-4d4f-b304-7719407468ff admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:50.598521 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 882/1749] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:48:50 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:48:50.599840 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e2ccb0-4d78-4341-a80c-d3d281b5581a admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:50.602976 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-f7e2ccb0-4d78-4341-a80c-d3d281b5581a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:50.627986 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e2ccb0-4d78-4341-a80c-d3d281b5581a admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:50.672033 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442fbafc-7754-4885-8d43-f64f4bd54d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:50.677677 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442fbafc-7754-4885-8d43-f64f4bd54d53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:50.678136 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442fbafc-7754-4885-8d43-f64f4bd54d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:50.678639 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442fbafc-7754-4885-8d43-f64f4bd54d53 admin admin] PATH_INFO: `/v3/projects/77a02e4255524676bcbc5e4ef5a018b8` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:50.680907 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442fbafc-7754-4885-8d43-f64f4bd54d53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77a02e4255524676bcbc5e4ef5a018b8)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:50.688388 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442fbafc-7754-4885-8d43-f64f4bd54d53 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:50.885416 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-d81ec421-1cbf-4d4f-b304-7719407468ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00a715aebe514121866463e092d66fa5', 'request_id': 'req-d81ec421-1cbf-4d4f-b304-7719407468ff'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:51.055896 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-442fbafc-7754-4885-8d43-f64f4bd54d53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77a02e4255524676bcbc5e4ef5a018b8', 'request_id': 'req-442fbafc-7754-4885-8d43-f64f4bd54d53'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:51.084557 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-442fbafc-7754-4885-8d43-f64f4bd54d53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77a02e4255524676bcbc5e4ef5a018b8', 'request_id': 'req-442fbafc-7754-4885-8d43-f64f4bd54d53'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:51.088092 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-d81ec421-1cbf-4d4f-b304-7719407468ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00a715aebe514121866463e092d66fa5', 'request_id': 'req-d81ec421-1cbf-4d4f-b304-7719407468ff'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:51.099542 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-d81ec421-1cbf-4d4f-b304-7719407468ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00a715aebe514121866463e092d66fa5', 'request_id': 'req-d81ec421-1cbf-4d4f-b304-7719407468ff'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:51.136629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-442fbafc-7754-4885-8d43-f64f4bd54d53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77a02e4255524676bcbc5e4ef5a018b8', 'request_id': 'req-442fbafc-7754-4885-8d43-f64f4bd54d53'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:51.161920 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-d81ec421-1cbf-4d4f-b304-7719407468ff admin admin] The token cache is being invalidate because project 00a715aebe514121866463e092d66fa5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:48:51.162220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-d81ec421-1cbf-4d4f-b304-7719407468ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:51.162941 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-d81ec421-1cbf-4d4f-b304-7719407468ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:51.164841 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 868/1750] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:48:50 2022] DELETE /identity/v3/projects/00a715aebe514121866463e092d66fa5 => generated 0 bytes in 687 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:48:51.165569 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81ec421-1cbf-4d4f-b304-7719407468ff admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:51.169572 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-d81ec421-1cbf-4d4f-b304-7719407468ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:51.209123 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:51.464084 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81ec421-1cbf-4d4f-b304-7719407468ff admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:51.465286 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-442fbafc-7754-4885-8d43-f64f4bd54d53 admin admin] The token cache is being invalidate because project 77a02e4255524676bcbc5e4ef5a018b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:48:51.465286 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-442fbafc-7754-4885-8d43-f64f4bd54d53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:51.465286 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-442fbafc-7754-4885-8d43-f64f4bd54d53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:48:51.465286 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 883/1751] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:48:50 2022] DELETE /identity/v3/projects/77a02e4255524676bcbc5e4ef5a018b8 => generated 0 bytes in 809 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:48:51.465286 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ff616-f293-46db-b3ab-649b9401bc9d None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:51.466278 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ff616-f293-46db-b3ab-649b9401bc9d None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:51.466278 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ff616-f293-46db-b3ab-649b9401bc9d None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:51.466278 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 884/1752] 10.222.0.52 () {54 vars in 921 bytes} [Mon Aug 29 22:48:51 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:48:51.466278 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd558316-00a9-49b7-ac63-fa8091371a3d None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:51.466278 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd558316-00a9-49b7-ac63-fa8091371a3d None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:51.466278 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd558316-00a9-49b7-ac63-fa8091371a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:51.477144 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-bd558316-00a9-49b7-ac63-fa8091371a3d None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:51.479973 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-bd558316-00a9-49b7-ac63-fa8091371a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:51.507306 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e63293a-a051-4632-b03d-eb72ec89bf90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:51.508920 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63293a-a051-4632-b03d-eb72ec89bf90 admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:51.511128 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63293a-a051-4632-b03d-eb72ec89bf90 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:51.511564 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63293a-a051-4632-b03d-eb72ec89bf90 admin admin] PATH_INFO: `/v3/projects/eb4d3967835b4c97bb12da77d1b5946f/users/2e00a82541ae4484b05fa765df939b33/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:51.523758 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:51.552793 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e63293a-a051-4632-b03d-eb72ec89bf90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb4d3967835b4c97bb12da77d1b5946f, user_id=2e00a82541ae4484b05fa765df939b33, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:51.556417 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e63293a-a051-4632-b03d-eb72ec89bf90 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:51.677292 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 885/1753] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:51.678977 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd760f7b-1d3a-4c7a-b0ab-1ee1acf507bc None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:51.679220 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd760f7b-1d3a-4c7a-b0ab-1ee1acf507bc None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:51.679480 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd760f7b-1d3a-4c7a-b0ab-1ee1acf507bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:51.699698 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 869/1754] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:48:51 2022] PUT /identity/v3/projects/eb4d3967835b4c97bb12da77d1b5946f/users/2e00a82541ae4484b05fa765df939b33/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:48:51.701674 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf2b55-4227-4e29-975d-093f2e521dd5 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:51.704033 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf2b55-4227-4e29-975d-093f2e521dd5 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:51.705239 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf2b55-4227-4e29-975d-093f2e521dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:51.737951 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-cd760f7b-1d3a-4c7a-b0ab-1ee1acf507bc None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:51.740551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-cd760f7b-1d3a-4c7a-b0ab-1ee1acf507bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:51.779718 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-02cf2b55-4227-4e29-975d-093f2e521dd5 None None] MFA Rules not processed for user `bdbf961e02934e88af3a72d6218f32fa`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:51.784502 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-02cf2b55-4227-4e29-975d-093f2e521dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:51.922285 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 886/1755] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:51.924203 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de5c42-72c1-4677-b48c-03ef072672b5 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:51.924505 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de5c42-72c1-4677-b48c-03ef072672b5 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:51.924816 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de5c42-72c1-4677-b48c-03ef072672b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:51.938974 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 870/1756] 10.222.0.52 () {58 vars in 1018 bytes} [Mon Aug 29 22:48:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:51.940165 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf2b55-4227-4e29-975d-093f2e521dd5 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:51.941809 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-02cf2b55-4227-4e29-975d-093f2e521dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:51.958281 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf2b55-4227-4e29-975d-093f2e521dd5 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:51.967700 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c9ff2-d530-467b-8578-dbf8962d3417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:51.968385 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c9ff2-d530-467b-8578-dbf8962d3417 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:51.968865 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c9ff2-d530-467b-8578-dbf8962d3417 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:51.973246 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c9ff2-d530-467b-8578-dbf8962d3417 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:51.973246 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c9ff2-d530-467b-8578-dbf8962d3417 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:51.973246 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c9ff2-d530-467b-8578-dbf8962d3417 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:52.002021 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 871/1757] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:48:51 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:48:52.002178 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-a6de5c42-72c1-4677-b48c-03ef072672b5 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:52.003734 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945303bb-4ed7-4746-96f0-b2539ade3128 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:52.004237 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-a6de5c42-72c1-4677-b48c-03ef072672b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:52.005086 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945303bb-4ed7-4746-96f0-b2539ade3128 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:52.005501 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945303bb-4ed7-4746-96f0-b2539ade3128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:52.026826 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 887/1758] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:52.030476 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de5c42-72c1-4677-b48c-03ef072672b5 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:52.045446 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-945303bb-4ed7-4746-96f0-b2539ade3128 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:52.047300 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-945303bb-4ed7-4746-96f0-b2539ade3128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:52.057429 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de5c42-72c1-4677-b48c-03ef072672b5 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:52.072871 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 872/1759] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:52.072871 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945303bb-4ed7-4746-96f0-b2539ade3128 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:52.075612 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25caa15-b900-4755-ad49-b5ad21ca1e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:52.081852 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25caa15-b900-4755-ad49-b5ad21ca1e7d admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:52.082428 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25caa15-b900-4755-ad49-b5ad21ca1e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:52.089028 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25caa15-b900-4755-ad49-b5ad21ca1e7d admin admin] PATH_INFO: `/v3/projects/eb4d3967835b4c97bb12da77d1b5946f/users/2e00a82541ae4484b05fa765df939b33/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:52.092534 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25caa15-b900-4755-ad49-b5ad21ca1e7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb4d3967835b4c97bb12da77d1b5946f, user_id=2e00a82541ae4484b05fa765df939b33, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:52.095804 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25caa15-b900-4755-ad49-b5ad21ca1e7d admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:52.099354 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945303bb-4ed7-4746-96f0-b2539ade3128 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:52.105292 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c714265-f02c-4a87-a55b-fd1b96863984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:52.106242 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c714265-f02c-4a87-a55b-fd1b96863984 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:52.106601 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c714265-f02c-4a87-a55b-fd1b96863984 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:52.107056 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c714265-f02c-4a87-a55b-fd1b96863984 admin admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:52.107791 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c714265-f02c-4a87-a55b-fd1b96863984 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:48:52.108315 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c714265-f02c-4a87-a55b-fd1b96863984 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:52.110724 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c714265-f02c-4a87-a55b-fd1b96863984 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:52.118289 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 888/1760] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:48:52 2022] PUT /identity/v3/projects/eb4d3967835b4c97bb12da77d1b5946f/users/2e00a82541ae4484b05fa765df939b33/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:48:52.121120 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25caa15-b900-4755-ad49-b5ad21ca1e7d admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:52.126895 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 873/1761] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:48:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:48:52.131513 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c714265-f02c-4a87-a55b-fd1b96863984 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:52.154105 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c714265-f02c-4a87-a55b-fd1b96863984 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:52.298155 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25caa15-b900-4755-ad49-b5ad21ca1e7d admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:52.301566 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e6454a-1f38-4344-b614-f401c148cf46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:52.303112 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e6454a-1f38-4344-b614-f401c148cf46 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:52.304191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e6454a-1f38-4344-b614-f401c148cf46 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:52.304191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e6454a-1f38-4344-b614-f401c148cf46 admin admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:52.304702 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e6454a-1f38-4344-b614-f401c148cf46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:52.307031 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e6454a-1f38-4344-b614-f401c148cf46 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:52.315117 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20675196-6756-4453-8aa1-9bb9bfba711e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:52.316197 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20675196-6756-4453-8aa1-9bb9bfba711e admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:52.316613 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20675196-6756-4453-8aa1-9bb9bfba711e admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:52.316943 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20675196-6756-4453-8aa1-9bb9bfba711e admin admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:52.322111 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20675196-6756-4453-8aa1-9bb9bfba711e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:48:52.322111 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20675196-6756-4453-8aa1-9bb9bfba711e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:52.322111 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20675196-6756-4453-8aa1-9bb9bfba711e admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:52.345086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 889/1762] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:48:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:48:52.347755 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c0fc2-9f3c-40cc-92b4-3987cf63211d None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:52.348263 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c0fc2-9f3c-40cc-92b4-3987cf63211d None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:52.348706 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c0fc2-9f3c-40cc-92b4-3987cf63211d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:52.385837 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-a98c0fc2-9f3c-40cc-92b4-3987cf63211d None None] MFA Rules not processed for user `2e00a82541ae4484b05fa765df939b33`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:52.387444 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-a98c0fc2-9f3c-40cc-92b4-3987cf63211d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:52.444445 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 874/1763] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:48:52 2022] POST /identity/v3/projects => generated 375 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:48:52.445485 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e6454a-1f38-4344-b614-f401c148cf46 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:52.513853 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 890/1764] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:52 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:52.516646 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98c0fc2-9f3c-40cc-92b4-3987cf63211d None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:52.518839 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-a98c0fc2-9f3c-40cc-92b4-3987cf63211d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:52.537352 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e6454a-1f38-4344-b614-f401c148cf46 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:52.544492 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41005a8b-9d0e-4b3e-a223-d9705d374c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:52.545210 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41005a8b-9d0e-4b3e-a223-d9705d374c0b admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:52.545556 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41005a8b-9d0e-4b3e-a223-d9705d374c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:52.545931 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41005a8b-9d0e-4b3e-a223-d9705d374c0b admin admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:52.546700 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41005a8b-9d0e-4b3e-a223-d9705d374c0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:52.548792 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41005a8b-9d0e-4b3e-a223-d9705d374c0b admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:52.611886 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98c0fc2-9f3c-40cc-92b4-3987cf63211d None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:52.666381 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 875/1765] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:48:52 2022] POST /identity/v3/projects => generated 383 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:48:52.666381 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41005a8b-9d0e-4b3e-a223-d9705d374c0b admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:52.711995 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee0d3fe-bfb3-4c44-a168-4acf4871c152 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:52.712644 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0d3fe-bfb3-4c44-a168-4acf4871c152 service nova] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:52.712989 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0d3fe-bfb3-4c44-a168-4acf4871c152 service nova] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:52.715733 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0d3fe-bfb3-4c44-a168-4acf4871c152 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:52.730846 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee0d3fe-bfb3-4c44-a168-4acf4871c152 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:52.732578 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee0d3fe-bfb3-4c44-a168-4acf4871c152 service nova] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:52.745879 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 891/1766] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:48:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:48:52.746852 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee0d3fe-bfb3-4c44-a168-4acf4871c152 service nova] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:52.748284 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-2ee0d3fe-bfb3-4c44-a168-4acf4871c152 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:52.765855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41005a8b-9d0e-4b3e-a223-d9705d374c0b admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:52.770885 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e89b5c-f1c9-4ba4-ae36-53887c2ac499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:52.771590 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e89b5c-f1c9-4ba4-ae36-53887c2ac499 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:52.771946 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e89b5c-f1c9-4ba4-ae36-53887c2ac499 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:52.772290 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e89b5c-f1c9-4ba4-ae36-53887c2ac499 admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:52.773046 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e89b5c-f1c9-4ba4-ae36-53887c2ac499 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:52.775061 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e89b5c-f1c9-4ba4-ae36-53887c2ac499 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:52.843999 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee0d3fe-bfb3-4c44-a168-4acf4871c152 service nova] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:52.861076 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 876/1767] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:48:52 2022] POST /identity/v3/users => generated 288 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:48:52.864924 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e89b5c-f1c9-4ba4-ae36-53887c2ac499 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:52.885678 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e89b5c-f1c9-4ba4-ae36-53887c2ac499 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:52.891678 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926c5ff0-cb18-44d4-be06-c820c8dd0015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:52.892480 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c5ff0-cb18-44d4-be06-c820c8dd0015 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:52.892597 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c5ff0-cb18-44d4-be06-c820c8dd0015 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:52.892907 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c5ff0-cb18-44d4-be06-c820c8dd0015 admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:52.893507 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926c5ff0-cb18-44d4-be06-c820c8dd0015 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:52.895697 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926c5ff0-cb18-44d4-be06-c820c8dd0015 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:52.924366 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22508514-efaa-4e69-bca1-aff07a03376d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:52.925097 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22508514-efaa-4e69-bca1-aff07a03376d service cinder] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:52.925377 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22508514-efaa-4e69-bca1-aff07a03376d service cinder] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:52.925650 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22508514-efaa-4e69-bca1-aff07a03376d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:53.001505 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22508514-efaa-4e69-bca1-aff07a03376d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:53.003044 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22508514-efaa-4e69-bca1-aff07a03376d service cinder] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:53.022838 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 892/1768] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:48:52 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:48:53.023537 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22508514-efaa-4e69-bca1-aff07a03376d service cinder] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:53.037280 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22508514-efaa-4e69-bca1-aff07a03376d service cinder] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:53.043354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a89ed2-ba16-4ece-ae2c-d85e20b67e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:53.043561 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 877/1769] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:48:52 2022] POST /identity/v3/users => generated 292 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:48:53.044220 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a89ed2-ba16-4ece-ae2c-d85e20b67e6d admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:53.044360 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926c5ff0-cb18-44d4-be06-c820c8dd0015 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:53.044680 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a89ed2-ba16-4ece-ae2c-d85e20b67e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:53.045065 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a89ed2-ba16-4ece-ae2c-d85e20b67e6d admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:53.045876 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a89ed2-ba16-4ece-ae2c-d85e20b67e6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:53.047634 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a89ed2-ba16-4ece-ae2c-d85e20b67e6d admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:53.056803 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926c5ff0-cb18-44d4-be06-c820c8dd0015 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:53.069719 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 893/1770] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:48:53 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:48:53.073346 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a89ed2-ba16-4ece-ae2c-d85e20b67e6d admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:53.077480 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb2e49e-dcf2-49d2-9310-4ced7b9d5882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:53.081686 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb2e49e-dcf2-49d2-9310-4ced7b9d5882 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:53.081686 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb2e49e-dcf2-49d2-9310-4ced7b9d5882 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:53.081686 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb2e49e-dcf2-49d2-9310-4ced7b9d5882 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:53.090165 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb2e49e-dcf2-49d2-9310-4ced7b9d5882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:53.091319 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a89ed2-ba16-4ece-ae2c-d85e20b67e6d admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:53.094979 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb2e49e-dcf2-49d2-9310-4ced7b9d5882 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:53.098012 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817d1ba7-5f15-45f2-a440-3c615720c4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:53.098640 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d1ba7-5f15-45f2-a440-3c615720c4a9 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:53.098971 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d1ba7-5f15-45f2-a440-3c615720c4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:53.099587 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d1ba7-5f15-45f2-a440-3c615720c4a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:53.100518 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817d1ba7-5f15-45f2-a440-3c615720c4a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:53.102447 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817d1ba7-5f15-45f2-a440-3c615720c4a9 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:53.115379 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 878/1771] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:48:53 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:48:53.119590 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb2e49e-dcf2-49d2-9310-4ced7b9d5882 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:53.125051 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 894/1772] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:48:53 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:48:53.137499 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817d1ba7-5f15-45f2-a440-3c615720c4a9 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:53.146698 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb2e49e-dcf2-49d2-9310-4ced7b9d5882 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:53.153394 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817d1ba7-5f15-45f2-a440-3c615720c4a9 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:53.156551 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d2cd3-f215-4b00-ac87-e0edf223ab36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:53.157220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d2cd3-f215-4b00-ac87-e0edf223ab36 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:53.157600 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d2cd3-f215-4b00-ac87-e0edf223ab36 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:53.157794 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d2cd3-f215-4b00-ac87-e0edf223ab36 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:53.158468 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8d2cd3-f215-4b00-ac87-e0edf223ab36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:53.160035 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580be1b2-530a-45cd-86e6-34ce2d62697b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:53.160173 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8d2cd3-f215-4b00-ac87-e0edf223ab36 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:53.160858 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580be1b2-530a-45cd-86e6-34ce2d62697b admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:53.161204 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580be1b2-530a-45cd-86e6-34ce2d62697b admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:53.161547 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580be1b2-530a-45cd-86e6-34ce2d62697b admin admin] PATH_INFO: `/v3/projects/d937c852390d400dbf119ef1d8e7abd5/users/b4518e453f394aac84faae55322bb969/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:53.179605 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 879/1773] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:48:53 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:48:53.185344 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d2cd3-f215-4b00-ac87-e0edf223ab36 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:53.197555 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580be1b2-530a-45cd-86e6-34ce2d62697b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d937c852390d400dbf119ef1d8e7abd5, user_id=b4518e453f394aac84faae55322bb969, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:53.197841 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d2cd3-f215-4b00-ac87-e0edf223ab36 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:53.207107 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580be1b2-530a-45cd-86e6-34ce2d62697b admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:53.215536 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56cda1-1705-4db1-9a73-7a06f3d856e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:53.221395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56cda1-1705-4db1-9a73-7a06f3d856e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:53.221395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56cda1-1705-4db1-9a73-7a06f3d856e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:53.221395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56cda1-1705-4db1-9a73-7a06f3d856e5 admin admin] PATH_INFO: `/v3/projects/69b6daa432644afc90047d9cb7164c4c/users/ce59a8ef6c3640da8218defd5fbf2a76/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:53.243163 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b56cda1-1705-4db1-9a73-7a06f3d856e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69b6daa432644afc90047d9cb7164c4c, user_id=ce59a8ef6c3640da8218defd5fbf2a76, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:53.246249 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b56cda1-1705-4db1-9a73-7a06f3d856e5 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:53.291887 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 895/1774] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:48:53 2022] PUT /identity/v3/projects/d937c852390d400dbf119ef1d8e7abd5/users/b4518e453f394aac84faae55322bb969/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:48:53.304325 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1864cb-29a4-4eee-9f91-a1f7b19ffaea None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:53.304672 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1864cb-29a4-4eee-9f91-a1f7b19ffaea None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:53.304949 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1864cb-29a4-4eee-9f91-a1f7b19ffaea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:53.334097 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 880/1775] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:48:53 2022] PUT /identity/v3/projects/69b6daa432644afc90047d9cb7164c4c/users/ce59a8ef6c3640da8218defd5fbf2a76/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:48:53.341502 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9fcd68-8ab3-41e9-87c4-8e3cfbe66bc5 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:53.341502 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9fcd68-8ab3-41e9-87c4-8e3cfbe66bc5 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:53.341502 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9fcd68-8ab3-41e9-87c4-8e3cfbe66bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:53.392051 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-ab1864cb-29a4-4eee-9f91-a1f7b19ffaea None None] MFA Rules not processed for user `b4518e453f394aac84faae55322bb969`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:53.397111 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-ab1864cb-29a4-4eee-9f91-a1f7b19ffaea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:53.407613 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-3b9fcd68-8ab3-41e9-87c4-8e3cfbe66bc5 None None] MFA Rules not processed for user `ce59a8ef6c3640da8218defd5fbf2a76`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:53.409360 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-3b9fcd68-8ab3-41e9-87c4-8e3cfbe66bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:53.499270 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:53.525060 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 896/1776] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:53.526401 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1864cb-29a4-4eee-9f91-a1f7b19ffaea None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:53.541774 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 881/1777] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:48:53.544071 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9fcd68-8ab3-41e9-87c4-8e3cfbe66bc5 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:53.593325 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:53.670335 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1864cb-29a4-4eee-9f91-a1f7b19ffaea None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:53.679683 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8b059f-2de9-47ac-9d88-8358ca4af9cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:53.680346 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8b059f-2de9-47ac-9d88-8358ca4af9cd service cinder] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:53.680627 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8b059f-2de9-47ac-9d88-8358ca4af9cd service cinder] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:53.680989 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8b059f-2de9-47ac-9d88-8358ca4af9cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:53.684908 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-6b8b059f-2de9-47ac-9d88-8358ca4af9cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:53.685752 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9fcd68-8ab3-41e9-87c4-8e3cfbe66bc5 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:53.692190 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7b8fb-f4df-4070-a555-bdea08134387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:53.692871 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7b8fb-f4df-4070-a555-bdea08134387 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:53.693153 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7b8fb-f4df-4070-a555-bdea08134387 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:53.693419 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7b8fb-f4df-4070-a555-bdea08134387 admin admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:53.694132 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca7b8fb-f4df-4070-a555-bdea08134387 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:53.697460 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca7b8fb-f4df-4070-a555-bdea08134387 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:53.804844 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8b059f-2de9-47ac-9d88-8358ca4af9cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:53.806416 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8b059f-2de9-47ac-9d88-8358ca4af9cd service cinder] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:53.825149 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 897/1778] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:48:53 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:48:53.827188 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8b059f-2de9-47ac-9d88-8358ca4af9cd service cinder] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:53.840980 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8b059f-2de9-47ac-9d88-8358ca4af9cd service cinder] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:53.847886 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d7b14-4159-44df-8850-7ab48ef7e222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:53.848764 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d7b14-4159-44df-8850-7ab48ef7e222 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:53.848944 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d7b14-4159-44df-8850-7ab48ef7e222 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:53.849681 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d7b14-4159-44df-8850-7ab48ef7e222 admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:53.850152 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56d7b14-4159-44df-8850-7ab48ef7e222 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:53.855214 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56d7b14-4159-44df-8850-7ab48ef7e222 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:53.860046 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 882/1779] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:48:53 2022] POST /identity/v3/projects => generated 377 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:48:53.866360 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7b8fb-f4df-4070-a555-bdea08134387 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:53.966540 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 898/1780] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:48:53 2022] POST /identity/v3/projects => generated 383 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:48:53.977657 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d7b14-4159-44df-8850-7ab48ef7e222 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:54.098434 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7b8fb-f4df-4070-a555-bdea08134387 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:54.216379 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d7b14-4159-44df-8850-7ab48ef7e222 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:54.228482 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225ae1d5-0272-4a53-bf27-a1b6e4a077a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:54.229610 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ae1d5-0272-4a53-bf27-a1b6e4a077a7 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:54.229946 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ae1d5-0272-4a53-bf27-a1b6e4a077a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:54.230220 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ae1d5-0272-4a53-bf27-a1b6e4a077a7 admin admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:54.230873 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225ae1d5-0272-4a53-bf27-a1b6e4a077a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:54.234762 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225ae1d5-0272-4a53-bf27-a1b6e4a077a7 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:54.245089 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1e88fd-7106-4f1c-83e7-46548b3efe3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:54.245715 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e88fd-7106-4f1c-83e7-46548b3efe3a admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:54.245994 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e88fd-7106-4f1c-83e7-46548b3efe3a admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:54.246406 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e88fd-7106-4f1c-83e7-46548b3efe3a admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:54.247056 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1e88fd-7106-4f1c-83e7-46548b3efe3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:54.249097 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1e88fd-7106-4f1c-83e7-46548b3efe3a admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:54.368855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 883/1781] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:48:53 2022] POST /identity/v3/users => generated 295 bytes in 506 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:48:54.372380 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1e88fd-7106-4f1c-83e7-46548b3efe3a admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:54.381115 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 899/1782] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:48:53 2022] POST /identity/v3/users => generated 298 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:48:54.385398 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225ae1d5-0272-4a53-bf27-a1b6e4a077a7 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:54.406525 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225ae1d5-0272-4a53-bf27-a1b6e4a077a7 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:54.410919 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1e88fd-7106-4f1c-83e7-46548b3efe3a admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:54.411890 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e1e070-3c2a-4d82-a350-85bfbf40a46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:54.413725 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e1e070-3c2a-4d82-a350-85bfbf40a46a admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:54.413725 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e1e070-3c2a-4d82-a350-85bfbf40a46a admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:54.413725 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e1e070-3c2a-4d82-a350-85bfbf40a46a admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:54.414722 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e1e070-3c2a-4d82-a350-85bfbf40a46a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:54.417147 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e1e070-3c2a-4d82-a350-85bfbf40a46a admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:54.425360 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5194698-523a-4f85-b687-372c72fcfe24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:54.425983 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5194698-523a-4f85-b687-372c72fcfe24 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:54.426265 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5194698-523a-4f85-b687-372c72fcfe24 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:54.426538 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5194698-523a-4f85-b687-372c72fcfe24 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:54.427191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5194698-523a-4f85-b687-372c72fcfe24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:54.428836 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5194698-523a-4f85-b687-372c72fcfe24 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:54.431324 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 900/1783] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:48:54.436350 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e1e070-3c2a-4d82-a350-85bfbf40a46a admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:54.443082 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 884/1784] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:48:54.444582 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5194698-523a-4f85-b687-372c72fcfe24 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:54.459775 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5194698-523a-4f85-b687-372c72fcfe24 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:54.464799 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e1e070-3c2a-4d82-a350-85bfbf40a46a admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:54.465594 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded91059-fb19-4850-bed4-81b4f4fe3074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:54.466465 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded91059-fb19-4850-bed4-81b4f4fe3074 admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:54.466921 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded91059-fb19-4850-bed4-81b4f4fe3074 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:54.467385 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded91059-fb19-4850-bed4-81b4f4fe3074 admin admin] PATH_INFO: `/v3/projects/170d841910bf4f0aa0d1f6ac00ebf488/users/92feef7187da4ffe8f292de2036bc885/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:54.473353 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcca707-1963-4dd6-9ae1-733631b61d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:54.476852 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcca707-1963-4dd6-9ae1-733631b61d82 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:54.476852 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcca707-1963-4dd6-9ae1-733631b61d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:54.476852 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcca707-1963-4dd6-9ae1-733631b61d82 admin admin] PATH_INFO: `/v3/projects/5fb8b4901ec747829f20390a6441f140/users/45610afe1e894ccf9c84eb03ee4a4d1f/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:54.490208 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded91059-fb19-4850-bed4-81b4f4fe3074 admin admin] RBAC: Authorizing `identity:create_grant(project_id=170d841910bf4f0aa0d1f6ac00ebf488, user_id=92feef7187da4ffe8f292de2036bc885, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:54.493537 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded91059-fb19-4850-bed4-81b4f4fe3074 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:54.499955 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcca707-1963-4dd6-9ae1-733631b61d82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fb8b4901ec747829f20390a6441f140, user_id=45610afe1e894ccf9c84eb03ee4a4d1f, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:54.505793 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcca707-1963-4dd6-9ae1-733631b61d82 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:54.546718 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:54.576588 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 885/1786] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:48:54 2022] PUT /identity/v3/projects/170d841910bf4f0aa0d1f6ac00ebf488/users/92feef7187da4ffe8f292de2036bc885/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:48:54.580025 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 901/1786] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:48:54 2022] PUT /identity/v3/projects/5fb8b4901ec747829f20390a6441f140/users/45610afe1e894ccf9c84eb03ee4a4d1f/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:48:54.581654 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded91059-fb19-4850-bed4-81b4f4fe3074 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:54.581784 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcca707-1963-4dd6-9ae1-733631b61d82 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:54.659961 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded91059-fb19-4850-bed4-81b4f4fe3074 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:54.666027 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb58f1e-76c9-49e3-a6b7-7cad3eec5e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:54.666712 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb58f1e-76c9-49e3-a6b7-7cad3eec5e5b admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:54.666999 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb58f1e-76c9-49e3-a6b7-7cad3eec5e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:54.667292 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb58f1e-76c9-49e3-a6b7-7cad3eec5e5b admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:54.667960 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb58f1e-76c9-49e3-a6b7-7cad3eec5e5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:54.669668 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb58f1e-76c9-49e3-a6b7-7cad3eec5e5b admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:54.670034 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcca707-1963-4dd6-9ae1-733631b61d82 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:54.674938 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db81945c-b040-47d6-9f09-774cd2642d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:54.675599 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81945c-b040-47d6-9f09-774cd2642d9c admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:54.675832 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81945c-b040-47d6-9f09-774cd2642d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:54.676120 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81945c-b040-47d6-9f09-774cd2642d9c admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:54.676938 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db81945c-b040-47d6-9f09-774cd2642d9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:54.678609 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db81945c-b040-47d6-9f09-774cd2642d9c admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:54.681563 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 886/1787] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:48:54.685858 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb58f1e-76c9-49e3-a6b7-7cad3eec5e5b admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:54.693145 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 902/1788] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:48:54.697271 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db81945c-b040-47d6-9f09-774cd2642d9c admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:54.697909 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb58f1e-76c9-49e3-a6b7-7cad3eec5e5b admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:54.703655 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eea06ab-a2a5-4da7-be3e-13240fe30f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:54.705063 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea06ab-a2a5-4da7-be3e-13240fe30f77 admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:54.709096 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db81945c-b040-47d6-9f09-774cd2642d9c admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:54.709302 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea06ab-a2a5-4da7-be3e-13240fe30f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:54.709731 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea06ab-a2a5-4da7-be3e-13240fe30f77 admin admin] PATH_INFO: `/v3/projects/5fb8b4901ec747829f20390a6441f140/users/45610afe1e894ccf9c84eb03ee4a4d1f/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:54.713762 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eea06ab-a2a5-4da7-be3e-13240fe30f77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fb8b4901ec747829f20390a6441f140, user_id=45610afe1e894ccf9c84eb03ee4a4d1f, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:54.714414 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f57b9e5-c3e3-4cae-b4ba-1a3481cd0fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:54.715216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57b9e5-c3e3-4cae-b4ba-1a3481cd0fe7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:54.715604 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57b9e5-c3e3-4cae-b4ba-1a3481cd0fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:54.715987 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57b9e5-c3e3-4cae-b4ba-1a3481cd0fe7 admin admin] PATH_INFO: `/v3/projects/170d841910bf4f0aa0d1f6ac00ebf488/users/92feef7187da4ffe8f292de2036bc885/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:54.718360 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eea06ab-a2a5-4da7-be3e-13240fe30f77 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:54.718641 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f57b9e5-c3e3-4cae-b4ba-1a3481cd0fe7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=170d841910bf4f0aa0d1f6ac00ebf488, user_id=92feef7187da4ffe8f292de2036bc885, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:54.721682 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f57b9e5-c3e3-4cae-b4ba-1a3481cd0fe7 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:54.735846 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 903/1789] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:48:54 2022] PUT /identity/v3/projects/170d841910bf4f0aa0d1f6ac00ebf488/users/92feef7187da4ffe8f292de2036bc885/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:48:54.745188 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 887/1790] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:48:54 2022] PUT /identity/v3/projects/5fb8b4901ec747829f20390a6441f140/users/45610afe1e894ccf9c84eb03ee4a4d1f/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:48:54.746633 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e8afb-723f-44b0-9451-798a88bf6cf7 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:54.747273 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e8afb-723f-44b0-9451-798a88bf6cf7 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:54.747273 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e8afb-723f-44b0-9451-798a88bf6cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:54.766055 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29519d92-8271-4b77-9ed6-b6bb805b3667 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:54.766338 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29519d92-8271-4b77-9ed6-b6bb805b3667 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:54.766669 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29519d92-8271-4b77-9ed6-b6bb805b3667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:54.799076 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-8e5e8afb-723f-44b0-9451-798a88bf6cf7 None None] MFA Rules not processed for user `92feef7187da4ffe8f292de2036bc885`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:54.800515 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-8e5e8afb-723f-44b0-9451-798a88bf6cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:54.803278 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-29519d92-8271-4b77-9ed6-b6bb805b3667 None None] MFA Rules not processed for user `45610afe1e894ccf9c84eb03ee4a4d1f`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:54.804608 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-29519d92-8271-4b77-9ed6-b6bb805b3667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:54.901968 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 888/1791] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:54 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:54.909148 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 904/1792] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:48:54 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:54.922771 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29519d92-8271-4b77-9ed6-b6bb805b3667 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:54.924213 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-29519d92-8271-4b77-9ed6-b6bb805b3667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:54.924369 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5e8afb-723f-44b0-9451-798a88bf6cf7 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:54.965640 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:54.994490 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29519d92-8271-4b77-9ed6-b6bb805b3667 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:54.996299 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5e8afb-723f-44b0-9451-798a88bf6cf7 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:55.000421 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3433c5e9-d857-4e46-b322-5d990e9fbd15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:55.000625 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7477ff8b-4b06-4437-beaa-01859ea7f9de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:55.001051 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3433c5e9-d857-4e46-b322-5d990e9fbd15 service cinder] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:55.001276 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7477ff8b-4b06-4437-beaa-01859ea7f9de service cinder] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:55.001384 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3433c5e9-d857-4e46-b322-5d990e9fbd15 service cinder] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:55.001564 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7477ff8b-4b06-4437-beaa-01859ea7f9de service cinder] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:55.001672 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3433c5e9-d857-4e46-b322-5d990e9fbd15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:55.001843 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7477ff8b-4b06-4437-beaa-01859ea7f9de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:55.088369 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7477ff8b-4b06-4437-beaa-01859ea7f9de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:55.090747 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7477ff8b-4b06-4437-beaa-01859ea7f9de service cinder] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:55.106999 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 889/1793] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:48:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:48:55.110582 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3433c5e9-d857-4e46-b322-5d990e9fbd15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:55.112298 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3433c5e9-d857-4e46-b322-5d990e9fbd15 service cinder] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:55.123207 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 905/1794] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:48:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:48:56.786468 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:56.969730 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:57.746148 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:59.340387 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:59.443181 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:00.087556 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:00.363471 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:01.913458 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec27ba-3a94-4123-97e9-cdd431036338 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:01.914325 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec27ba-3a94-4123-97e9-cdd431036338 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:01.914460 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec27ba-3a94-4123-97e9-cdd431036338 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:01.916084 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 890/1795] 10.222.0.52 () {54 vars in 921 bytes} [Mon Aug 29 22:49:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:49:01.930063 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1619e01f-e104-436d-9463-66d6fddf9302 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:01.930640 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1619e01f-e104-436d-9463-66d6fddf9302 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:01.930640 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1619e01f-e104-436d-9463-66d6fddf9302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:01.955088 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-1619e01f-e104-436d-9463-66d6fddf9302 None None] MFA Rules not processed for user `bdbf961e02934e88af3a72d6218f32fa`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:49:01.956473 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-1619e01f-e104-436d-9463-66d6fddf9302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:02.021509 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:02.072873 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 906/1796] 10.222.0.52 () {58 vars in 1018 bytes} [Mon Aug 29 22:49:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:49:02.094041 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec27ba-3a94-4123-97e9-cdd431036338 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:02.138445 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec27ba-3a94-4123-97e9-cdd431036338 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:02.147991 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775d8f62-f0bd-4701-a4dc-d9166cb9e7b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:02.148667 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d8f62-f0bd-4701-a4dc-d9166cb9e7b1 service nova] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:02.149079 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d8f62-f0bd-4701-a4dc-d9166cb9e7b1 service nova] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:02.149228 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d8f62-f0bd-4701-a4dc-d9166cb9e7b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:02.161989 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775d8f62-f0bd-4701-a4dc-d9166cb9e7b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:02.164418 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775d8f62-f0bd-4701-a4dc-d9166cb9e7b1 service nova] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:02.174430 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:02.190972 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1619e01f-e104-436d-9463-66d6fddf9302 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:02.201988 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 891/1797] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:49:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:49:02.203958 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1619e01f-e104-436d-9463-66d6fddf9302 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:02.209179 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a413e-a7cd-4274-ad00-1db93e2e9c05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:02.209586 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a413e-a7cd-4274-ad00-1db93e2e9c05 service cinder] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:02.209957 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a413e-a7cd-4274-ad00-1db93e2e9c05 service cinder] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:02.210328 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a413e-a7cd-4274-ad00-1db93e2e9c05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:02.232698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50a413e-a7cd-4274-ad00-1db93e2e9c05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:02.234773 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50a413e-a7cd-4274-ad00-1db93e2e9c05 service cinder] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:02.247187 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 907/1798] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:49:02 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:49:02.424180 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775d8f62-f0bd-4701-a4dc-d9166cb9e7b1 service nova] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:02.425694 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-775d8f62-f0bd-4701-a4dc-d9166cb9e7b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:02.606337 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775d8f62-f0bd-4701-a4dc-d9166cb9e7b1 service nova] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:02.617488 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e657390-0968-4fed-aa95-814282e5f642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:02.618154 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e657390-0968-4fed-aa95-814282e5f642 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:02.618443 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e657390-0968-4fed-aa95-814282e5f642 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:02.618720 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e657390-0968-4fed-aa95-814282e5f642 admin admin] PATH_INFO: `/v3/users/2a5fc4b25bcc4c4298c82c7950033a9a` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:02.622408 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e657390-0968-4fed-aa95-814282e5f642 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a5fc4b25bcc4c4298c82c7950033a9a)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:02.624500 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e657390-0968-4fed-aa95-814282e5f642 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:02.899710 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-7e657390-0968-4fed-aa95-814282e5f642 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a5fc4b25bcc4c4298c82c7950033a9a'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:02.917969 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-7e657390-0968-4fed-aa95-814282e5f642 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a5fc4b25bcc4c4298c82c7950033a9a'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:02.929118 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-7e657390-0968-4fed-aa95-814282e5f642 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a5fc4b25bcc4c4298c82c7950033a9a'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:03.031611 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-7e657390-0968-4fed-aa95-814282e5f642 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a5fc4b25bcc4c4298c82c7950033a9a'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:03.033148 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-7e657390-0968-4fed-aa95-814282e5f642 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a5fc4b25bcc4c4298c82c7950033a9a'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:03.040547 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 892/1799] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:49:02 2022] DELETE /identity/v3/users/2a5fc4b25bcc4c4298c82c7950033a9a => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:49:03.049556 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ad51e-99a8-4641-b51d-5f3cec50f537 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:03.050102 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ad51e-99a8-4641-b51d-5f3cec50f537 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:03.050102 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ad51e-99a8-4641-b51d-5f3cec50f537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:03.074927 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-d01ad51e-99a8-4641-b51d-5f3cec50f537 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:49:03.074927 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-d01ad51e-99a8-4641-b51d-5f3cec50f537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:03.168211 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e657390-0968-4fed-aa95-814282e5f642 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:03.169889 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-7e657390-0968-4fed-aa95-814282e5f642 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:03.181394 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 908/1800] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:49:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:49:03.285286 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e657390-0968-4fed-aa95-814282e5f642 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:03.288085 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6c348-dedb-484a-907f-d6ca3210c832 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:03.289045 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6c348-dedb-484a-907f-d6ca3210c832 service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:03.289478 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6c348-dedb-484a-907f-d6ca3210c832 service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:03.289863 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6c348-dedb-484a-907f-d6ca3210c832 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:03.303689 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d6c348-dedb-484a-907f-d6ca3210c832 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:03.306353 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d6c348-dedb-484a-907f-d6ca3210c832 service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:03.322018 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 893/1801] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:49:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:49:03.683997 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ad51e-99a8-4641-b51d-5f3cec50f537 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:03.686075 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-d01ad51e-99a8-4641-b51d-5f3cec50f537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:03.785694 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ad51e-99a8-4641-b51d-5f3cec50f537 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:03.790023 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764ff0ea-1638-4f26-a31c-57625e3a89b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:03.790917 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ff0ea-1638-4f26-a31c-57625e3a89b9 service cinder] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:03.791302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ff0ea-1638-4f26-a31c-57625e3a89b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:03.791637 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ff0ea-1638-4f26-a31c-57625e3a89b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:03.793950 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-764ff0ea-1638-4f26-a31c-57625e3a89b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:03.887758 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764ff0ea-1638-4f26-a31c-57625e3a89b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:03.889566 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764ff0ea-1638-4f26-a31c-57625e3a89b9 service cinder] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:03.900526 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 909/1802] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:49:03 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:49:03.948192 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6c348-dedb-484a-907f-d6ca3210c832 service neutron] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:03.949724 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-b9d6c348-dedb-484a-907f-d6ca3210c832 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:03.974324 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6c348-dedb-484a-907f-d6ca3210c832 service neutron] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:03.978540 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4616052-81e1-4798-ab81-4d6c179e6da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:03.979175 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4616052-81e1-4798-ab81-4d6c179e6da0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:03.979451 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4616052-81e1-4798-ab81-4d6c179e6da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:03.979723 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4616052-81e1-4798-ab81-4d6c179e6da0 admin admin] PATH_INFO: `/v3/projects/144daa4319574a589302ec2383685566` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:03.980902 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4616052-81e1-4798-ab81-4d6c179e6da0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=144daa4319574a589302ec2383685566)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:03.983024 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4616052-81e1-4798-ab81-4d6c179e6da0 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:04.003001 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:04.108166 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-f4616052-81e1-4798-ab81-4d6c179e6da0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '144daa4319574a589302ec2383685566', 'request_id': 'req-f4616052-81e1-4798-ab81-4d6c179e6da0'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:04.248819 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-f4616052-81e1-4798-ab81-4d6c179e6da0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '144daa4319574a589302ec2383685566', 'request_id': 'req-f4616052-81e1-4798-ab81-4d6c179e6da0'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:04.255905 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-f4616052-81e1-4798-ab81-4d6c179e6da0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '144daa4319574a589302ec2383685566', 'request_id': 'req-f4616052-81e1-4798-ab81-4d6c179e6da0'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:04.267740 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-f4616052-81e1-4798-ab81-4d6c179e6da0 admin admin] The token cache is being invalidate because project 144daa4319574a589302ec2383685566 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:49:04.268045 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-f4616052-81e1-4798-ab81-4d6c179e6da0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:04.268707 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-f4616052-81e1-4798-ab81-4d6c179e6da0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:04.270271 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 894/1803] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:49:03 2022] DELETE /identity/v3/projects/144daa4319574a589302ec2383685566 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:49:04.275205 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e80203-0361-4265-b486-cfe2df670203 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:04.281962 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e80203-0361-4265-b486-cfe2df670203 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:04.281962 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e80203-0361-4265-b486-cfe2df670203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:04.296436 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-08e80203-0361-4265-b486-cfe2df670203 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:49:04.298278 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-08e80203-0361-4265-b486-cfe2df670203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:04.307934 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:04.361972 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 910/1804] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:49:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:49:04.369820 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e1172-c450-435d-8569-61d7043883db None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:04.370112 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e1172-c450-435d-8569-61d7043883db None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:04.370388 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e1172-c450-435d-8569-61d7043883db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:04.399356 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-956e1172-c450-435d-8569-61d7043883db None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:49:04.400552 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-956e1172-c450-435d-8569-61d7043883db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:04.409667 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 895/1805] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:49:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:49:04.414943 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e80203-0361-4265-b486-cfe2df670203 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:04.426859 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e80203-0361-4265-b486-cfe2df670203 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:04.435663 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187a0aa-e94b-460b-b889-9bb207ad0001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:04.436299 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187a0aa-e94b-460b-b889-9bb207ad0001 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:04.436576 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187a0aa-e94b-460b-b889-9bb207ad0001 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:04.436869 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187a0aa-e94b-460b-b889-9bb207ad0001 admin admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:04.437522 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4187a0aa-e94b-460b-b889-9bb207ad0001 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:49:04.437830 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4187a0aa-e94b-460b-b889-9bb207ad0001 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:04.439986 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4187a0aa-e94b-460b-b889-9bb207ad0001 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:04.458364 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 911/1806] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:49:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:49:04.458511 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e1172-c450-435d-8569-61d7043883db None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:04.469912 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e1172-c450-435d-8569-61d7043883db None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:04.475516 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6cdb22-a138-448d-bf44-ceb978ec7ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:04.479545 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6cdb22-a138-448d-bf44-ceb978ec7ba0 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:04.479545 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6cdb22-a138-448d-bf44-ceb978ec7ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:04.479545 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6cdb22-a138-448d-bf44-ceb978ec7ba0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:04.479545 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6cdb22-a138-448d-bf44-ceb978ec7ba0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:04.479545 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6cdb22-a138-448d-bf44-ceb978ec7ba0 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:04.596771 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 896/1807] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:49:04 2022] POST /identity/v3/projects => generated 405 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:49:04.608291 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187a0aa-e94b-460b-b889-9bb207ad0001 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:04.611891 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187a0aa-e94b-460b-b889-9bb207ad0001 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:04.670488 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc02ec1-d724-449d-9cd6-545d46fb5e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:04.671092 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc02ec1-d724-449d-9cd6-545d46fb5e33 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:04.671396 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc02ec1-d724-449d-9cd6-545d46fb5e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:04.671629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc02ec1-d724-449d-9cd6-545d46fb5e33 admin admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:04.672257 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc02ec1-d724-449d-9cd6-545d46fb5e33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:04.674888 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc02ec1-d724-449d-9cd6-545d46fb5e33 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:04.795572 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 912/1808] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:49:04 2022] POST /identity/v3/users => generated 303 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:49:04.800427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6cdb22-a138-448d-bf44-ceb978ec7ba0 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:04.810792 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6cdb22-a138-448d-bf44-ceb978ec7ba0 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:04.815963 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3112e052-1405-49d3-b20e-d27bb7833f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:04.816346 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3112e052-1405-49d3-b20e-d27bb7833f10 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:04.816537 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3112e052-1405-49d3-b20e-d27bb7833f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:04.816846 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3112e052-1405-49d3-b20e-d27bb7833f10 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:04.817493 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3112e052-1405-49d3-b20e-d27bb7833f10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:04.819093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3112e052-1405-49d3-b20e-d27bb7833f10 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:04.829949 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 897/1809] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:49:04.833951 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc02ec1-d724-449d-9cd6-545d46fb5e33 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:04.847790 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc02ec1-d724-449d-9cd6-545d46fb5e33 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:04.852697 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd832194-e089-4cd8-9548-166acc4c2d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:04.853346 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd832194-e089-4cd8-9548-166acc4c2d57 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:04.853619 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd832194-e089-4cd8-9548-166acc4c2d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:04.853885 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd832194-e089-4cd8-9548-166acc4c2d57 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:04.854527 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd832194-e089-4cd8-9548-166acc4c2d57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:04.856318 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd832194-e089-4cd8-9548-166acc4c2d57 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:04.874118 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 913/1810] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:49:04.878546 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3112e052-1405-49d3-b20e-d27bb7833f10 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:04.888827 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3112e052-1405-49d3-b20e-d27bb7833f10 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:04.895105 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212bae8c-4dc6-493b-8f84-ba8e58e4585c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:04.895784 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212bae8c-4dc6-493b-8f84-ba8e58e4585c admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:04.896063 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212bae8c-4dc6-493b-8f84-ba8e58e4585c admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:04.896333 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212bae8c-4dc6-493b-8f84-ba8e58e4585c admin admin] PATH_INFO: `/v3/projects/91b737cece7b45c098d6e48f683b786c/users/90f717da478246c48af95cdbee5bc81c/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:04.913496 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212bae8c-4dc6-493b-8f84-ba8e58e4585c admin admin] RBAC: Authorizing `identity:create_grant(project_id=91b737cece7b45c098d6e48f683b786c, user_id=90f717da478246c48af95cdbee5bc81c, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:04.916250 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212bae8c-4dc6-493b-8f84-ba8e58e4585c admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:04.998149 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 898/1811] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:49:04 2022] PUT /identity/v3/projects/91b737cece7b45c098d6e48f683b786c/users/90f717da478246c48af95cdbee5bc81c/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:49:05.003296 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dee0cb-fc62-40b2-be5d-125ebea5d582 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:05.003581 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dee0cb-fc62-40b2-be5d-125ebea5d582 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:05.003854 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dee0cb-fc62-40b2-be5d-125ebea5d582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:05.024637 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-03dee0cb-fc62-40b2-be5d-125ebea5d582 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:49:05.025924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-03dee0cb-fc62-40b2-be5d-125ebea5d582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:05.090315 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 914/1812] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:49:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:49:05.101214 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212bae8c-4dc6-493b-8f84-ba8e58e4585c admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:05.102950 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-212bae8c-4dc6-493b-8f84-ba8e58e4585c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:05.166950 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212bae8c-4dc6-493b-8f84-ba8e58e4585c admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:05.170526 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c2541a-d31c-46cd-833f-a27ba0dcb045 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:05.171135 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2541a-d31c-46cd-833f-a27ba0dcb045 service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:05.171413 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2541a-d31c-46cd-833f-a27ba0dcb045 service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:05.171683 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2541a-d31c-46cd-833f-a27ba0dcb045 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:05.182550 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c2541a-d31c-46cd-833f-a27ba0dcb045 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:05.184041 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c2541a-d31c-46cd-833f-a27ba0dcb045 service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:05.195523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 899/1813] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:49:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:49:06.530383 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd03715-4977-497c-b002-b2fe934b9a53 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:06.530862 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd03715-4977-497c-b002-b2fe934b9a53 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:06.530956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd03715-4977-497c-b002-b2fe934b9a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:06.567999 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-ddd03715-4977-497c-b002-b2fe934b9a53 None None] MFA Rules not processed for user `90f717da478246c48af95cdbee5bc81c`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:49:06.569297 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-ddd03715-4977-497c-b002-b2fe934b9a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:06.578834 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:06.663371 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 915/1814] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:49:06 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:49:06.667976 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c2541a-d31c-46cd-833f-a27ba0dcb045 service neutron] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:06.680335 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c2541a-d31c-46cd-833f-a27ba0dcb045 service neutron] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:06.684377 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43dbd4-ea89-4079-bc55-19e080969c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:06.685038 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43dbd4-ea89-4079-bc55-19e080969c3d admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:06.685700 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43dbd4-ea89-4079-bc55-19e080969c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:06.685818 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43dbd4-ea89-4079-bc55-19e080969c3d admin admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:06.686395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f43dbd4-ea89-4079-bc55-19e080969c3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:06.688319 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f43dbd4-ea89-4079-bc55-19e080969c3d admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:06.803982 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 900/1815] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:49:06 2022] POST /identity/v3/projects => generated 405 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:49:06.807996 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd03715-4977-497c-b002-b2fe934b9a53 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:06.820283 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd03715-4977-497c-b002-b2fe934b9a53 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:06.897176 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fbb160-5860-46a2-8e4e-1dd94b3ec16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:06.897972 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fbb160-5860-46a2-8e4e-1dd94b3ec16b admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:06.899031 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fbb160-5860-46a2-8e4e-1dd94b3ec16b admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:06.899031 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fbb160-5860-46a2-8e4e-1dd94b3ec16b admin admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:06.899506 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fbb160-5860-46a2-8e4e-1dd94b3ec16b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:06.901597 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fbb160-5860-46a2-8e4e-1dd94b3ec16b admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:06.977987 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 916/1816] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:49:06 2022] POST /identity/v3/users => generated 309 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:49:06.982532 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43dbd4-ea89-4079-bc55-19e080969c3d admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:06.996958 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43dbd4-ea89-4079-bc55-19e080969c3d admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:07.002838 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3c18f-8646-4687-8d3d-d33630fe8cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:07.003596 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3c18f-8646-4687-8d3d-d33630fe8cdf admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:07.003882 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3c18f-8646-4687-8d3d-d33630fe8cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:07.004259 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3c18f-8646-4687-8d3d-d33630fe8cdf admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:07.005043 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd3c18f-8646-4687-8d3d-d33630fe8cdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:07.006868 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd3c18f-8646-4687-8d3d-d33630fe8cdf admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:07.020322 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 901/1817] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:49:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:49:07.025133 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fbb160-5860-46a2-8e4e-1dd94b3ec16b admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:07.038207 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fbb160-5860-46a2-8e4e-1dd94b3ec16b admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:07.045260 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc5cd46-bf5d-4b08-b044-cfbd8749e973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:07.045785 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5cd46-bf5d-4b08-b044-cfbd8749e973 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:07.046067 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5cd46-bf5d-4b08-b044-cfbd8749e973 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:07.046350 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5cd46-bf5d-4b08-b044-cfbd8749e973 admin admin] PATH_INFO: `/v3/projects/646bce3324f44557841140de5f82e4ec/users/3c495f6bbb684553b48feb26619413d2/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:07.067137 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc5cd46-bf5d-4b08-b044-cfbd8749e973 admin admin] RBAC: Authorizing `identity:create_grant(project_id=646bce3324f44557841140de5f82e4ec, user_id=3c495f6bbb684553b48feb26619413d2, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:07.069875 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc5cd46-bf5d-4b08-b044-cfbd8749e973 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:07.145513 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 917/1818] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:49:07 2022] PUT /identity/v3/projects/646bce3324f44557841140de5f82e4ec/users/3c495f6bbb684553b48feb26619413d2/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:49:07.146925 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3c18f-8646-4687-8d3d-d33630fe8cdf admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:07.158542 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3c18f-8646-4687-8d3d-d33630fe8cdf admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:07.170611 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:07.229210 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4e9741-1a5f-44d5-9212-1f221d2d1049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:07.229832 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e9741-1a5f-44d5-9212-1f221d2d1049 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:07.230115 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e9741-1a5f-44d5-9212-1f221d2d1049 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:07.230391 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e9741-1a5f-44d5-9212-1f221d2d1049 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:07.231313 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4e9741-1a5f-44d5-9212-1f221d2d1049 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:07.234771 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4e9741-1a5f-44d5-9212-1f221d2d1049 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:07.246552 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 902/1819] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:49:07 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:49:07.251850 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc5cd46-bf5d-4b08-b044-cfbd8749e973 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:07.268503 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc5cd46-bf5d-4b08-b044-cfbd8749e973 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:07.276991 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca8ea6-de9d-470f-b14d-4f812e204e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:07.277989 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca8ea6-de9d-470f-b14d-4f812e204e45 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:07.278965 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca8ea6-de9d-470f-b14d-4f812e204e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:07.278965 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca8ea6-de9d-470f-b14d-4f812e204e45 admin admin] PATH_INFO: `/v3/projects/646bce3324f44557841140de5f82e4ec/users/3c495f6bbb684553b48feb26619413d2/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:07.282761 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca8ea6-de9d-470f-b14d-4f812e204e45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=646bce3324f44557841140de5f82e4ec, user_id=3c495f6bbb684553b48feb26619413d2, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:07.288898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca8ea6-de9d-470f-b14d-4f812e204e45 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:07.308871 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 918/1820] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:49:07 2022] PUT /identity/v3/projects/646bce3324f44557841140de5f82e4ec/users/3c495f6bbb684553b48feb26619413d2/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:49:08.788869 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc0a83-e706-4e0e-b6cd-a7395a207744 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:08.789451 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc0a83-e706-4e0e-b6cd-a7395a207744 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:08.789642 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc0a83-e706-4e0e-b6cd-a7395a207744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:08.831407 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-24bc0a83-e706-4e0e-b6cd-a7395a207744 None None] MFA Rules not processed for user `3c495f6bbb684553b48feb26619413d2`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:49:08.833554 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-24bc0a83-e706-4e0e-b6cd-a7395a207744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:08.963357 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 903/1821] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:49:08 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:49:09.997735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9284d-7aeb-4266-9bd1-0994c98f51de None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:09.998642 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9284d-7aeb-4266-9bd1-0994c98f51de None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:10.002615 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9284d-7aeb-4266-9bd1-0994c98f51de None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:10.009775 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 919/1822] 10.222.0.52 () {54 vars in 921 bytes} [Mon Aug 29 22:49:09 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:49:10.048878 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a5e8d2-9828-4827-a378-1a8abfedff14 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:10.049505 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a5e8d2-9828-4827-a378-1a8abfedff14 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:10.049505 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a5e8d2-9828-4827-a378-1a8abfedff14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:10.077887 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-25a5e8d2-9828-4827-a378-1a8abfedff14 None None] MFA Rules not processed for user `d8fd0f6e5b144d2ca78b8d96722502a7`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:49:10.079659 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-25a5e8d2-9828-4827-a378-1a8abfedff14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:10.158832 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 904/1823] 10.222.0.52 () {58 vars in 1018 bytes} [Mon Aug 29 22:49:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:49:10.169938 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe9284d-7aeb-4266-9bd1-0994c98f51de None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:10.171802 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-ebe9284d-7aeb-4266-9bd1-0994c98f51de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:10.189604 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe9284d-7aeb-4266-9bd1-0994c98f51de None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:10.194736 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df65f231-9804-43fd-86a0-d4a229cbd6f8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8fd0f6e5b144d2ca78b8d96722502a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:10.195409 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65f231-9804-43fd-86a0-d4a229cbd6f8 service placement] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:10.195869 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65f231-9804-43fd-86a0-d4a229cbd6f8 service placement] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:10.196399 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65f231-9804-43fd-86a0-d4a229cbd6f8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:10.210593 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df65f231-9804-43fd-86a0-d4a229cbd6f8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:10.214403 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df65f231-9804-43fd-86a0-d4a229cbd6f8 service placement] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:10.233369 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 920/1824] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:49:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:49:11.012242 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54576617-9a3c-455e-a019-110c26b25d1e None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:11.012536 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54576617-9a3c-455e-a019-110c26b25d1e None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:11.013102 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54576617-9a3c-455e-a019-110c26b25d1e None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:11.014893 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 905/1825] 10.222.0.52 () {54 vars in 921 bytes} [Mon Aug 29 22:49:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:49:11.025185 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b06c1-7278-46ac-bb0b-1672b1ade5cf None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:11.028927 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b06c1-7278-46ac-bb0b-1672b1ade5cf None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:11.028927 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b06c1-7278-46ac-bb0b-1672b1ade5cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:11.079850 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-082b06c1-7278-46ac-bb0b-1672b1ade5cf None None] MFA Rules not processed for user `d8fd0f6e5b144d2ca78b8d96722502a7`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:49:11.082493 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-082b06c1-7278-46ac-bb0b-1672b1ade5cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:11.093315 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 921/1826] 10.222.0.52 () {58 vars in 1018 bytes} [Mon Aug 29 22:49:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:49:11.105473 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54576617-9a3c-455e-a019-110c26b25d1e None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:11.119848 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54576617-9a3c-455e-a019-110c26b25d1e None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:11.125008 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431ab27-1cee-42f9-8f16-863cbb0a60cb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8fd0f6e5b144d2ca78b8d96722502a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:11.125680 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431ab27-1cee-42f9-8f16-863cbb0a60cb service placement] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:11.126622 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431ab27-1cee-42f9-8f16-863cbb0a60cb service placement] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:11.126949 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431ab27-1cee-42f9-8f16-863cbb0a60cb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:11.136299 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4431ab27-1cee-42f9-8f16-863cbb0a60cb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:11.137917 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4431ab27-1cee-42f9-8f16-863cbb0a60cb service placement] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:11.151887 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 906/1827] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:49:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:49:11.245186 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082b06c1-7278-46ac-bb0b-1672b1ade5cf None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:11.302324 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:11.325182 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082b06c1-7278-46ac-bb0b-1672b1ade5cf None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:11.331248 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01212671-f8ea-4a82-811e-36152a76386a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:11.331964 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01212671-f8ea-4a82-811e-36152a76386a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:11.332480 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01212671-f8ea-4a82-811e-36152a76386a admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:11.332841 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01212671-f8ea-4a82-811e-36152a76386a admin admin] PATH_INFO: `/v3/users/90f717da478246c48af95cdbee5bc81c` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:11.334275 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01212671-f8ea-4a82-811e-36152a76386a admin admin] RBAC: Authorizing `identity:delete_user(user_id=90f717da478246c48af95cdbee5bc81c)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:11.336587 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01212671-f8ea-4a82-811e-36152a76386a admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:11.582416 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-01212671-f8ea-4a82-811e-36152a76386a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90f717da478246c48af95cdbee5bc81c'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:11.654562 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-01212671-f8ea-4a82-811e-36152a76386a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90f717da478246c48af95cdbee5bc81c'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:11.668459 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-01212671-f8ea-4a82-811e-36152a76386a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90f717da478246c48af95cdbee5bc81c'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:11.675967 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-01212671-f8ea-4a82-811e-36152a76386a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90f717da478246c48af95cdbee5bc81c'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:11.676681 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-01212671-f8ea-4a82-811e-36152a76386a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90f717da478246c48af95cdbee5bc81c'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:11.682938 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 922/1828] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:49:11 2022] DELETE /identity/v3/users/90f717da478246c48af95cdbee5bc81c => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:49:11.976472 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431ab27-1cee-42f9-8f16-863cbb0a60cb service placement] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:11.994978 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431ab27-1cee-42f9-8f16-863cbb0a60cb service placement] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:12.063110 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:12.121759 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539325be-8f3d-48c2-9a16-e34deed09da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:12.122387 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539325be-8f3d-48c2-9a16-e34deed09da6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:12.122660 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539325be-8f3d-48c2-9a16-e34deed09da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:12.122933 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539325be-8f3d-48c2-9a16-e34deed09da6 admin admin] PATH_INFO: `/v3/projects/91b737cece7b45c098d6e48f683b786c` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:12.124074 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539325be-8f3d-48c2-9a16-e34deed09da6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91b737cece7b45c098d6e48f683b786c)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:12.126079 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539325be-8f3d-48c2-9a16-e34deed09da6 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:12.245568 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-539325be-8f3d-48c2-9a16-e34deed09da6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91b737cece7b45c098d6e48f683b786c', 'request_id': 'req-539325be-8f3d-48c2-9a16-e34deed09da6'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:12.332460 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-539325be-8f3d-48c2-9a16-e34deed09da6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91b737cece7b45c098d6e48f683b786c', 'request_id': 'req-539325be-8f3d-48c2-9a16-e34deed09da6'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:12.338786 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-539325be-8f3d-48c2-9a16-e34deed09da6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91b737cece7b45c098d6e48f683b786c', 'request_id': 'req-539325be-8f3d-48c2-9a16-e34deed09da6'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:12.350722 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-539325be-8f3d-48c2-9a16-e34deed09da6 admin admin] The token cache is being invalidate because project 91b737cece7b45c098d6e48f683b786c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:49:12.351016 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-539325be-8f3d-48c2-9a16-e34deed09da6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:12.351680 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-539325be-8f3d-48c2-9a16-e34deed09da6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:12.353548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 907/1829] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:49:11 2022] DELETE /identity/v3/projects/91b737cece7b45c098d6e48f683b786c => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:49:12.357902 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01212671-f8ea-4a82-811e-36152a76386a admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:12.363531 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-01212671-f8ea-4a82-811e-36152a76386a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:12.380781 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01212671-f8ea-4a82-811e-36152a76386a admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:12.453504 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d967455f-41ae-4560-a594-b1811962d995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:12.454191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d967455f-41ae-4560-a594-b1811962d995 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:12.454516 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d967455f-41ae-4560-a594-b1811962d995 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:12.454842 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d967455f-41ae-4560-a594-b1811962d995 admin admin] PATH_INFO: `/v3/users/3c495f6bbb684553b48feb26619413d2` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:12.456218 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d967455f-41ae-4560-a594-b1811962d995 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c495f6bbb684553b48feb26619413d2)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:12.458823 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d967455f-41ae-4560-a594-b1811962d995 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:12.646299 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d967455f-41ae-4560-a594-b1811962d995 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c495f6bbb684553b48feb26619413d2'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:12.717477 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d967455f-41ae-4560-a594-b1811962d995 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c495f6bbb684553b48feb26619413d2'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:12.725564 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:12.730772 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d967455f-41ae-4560-a594-b1811962d995 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c495f6bbb684553b48feb26619413d2'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:12.738233 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d967455f-41ae-4560-a594-b1811962d995 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c495f6bbb684553b48feb26619413d2'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:12.739786 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d967455f-41ae-4560-a594-b1811962d995 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c495f6bbb684553b48feb26619413d2'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:12.744738 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 923/1830] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:49:12 2022] DELETE /identity/v3/users/3c495f6bbb684553b48feb26619413d2 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:49:13.225720 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539325be-8f3d-48c2-9a16-e34deed09da6 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:13.228539 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-539325be-8f3d-48c2-9a16-e34deed09da6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:13.243213 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539325be-8f3d-48c2-9a16-e34deed09da6 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:13.317882 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585200dd-3255-4e04-8d00-4158c80b1ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:13.318540 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585200dd-3255-4e04-8d00-4158c80b1ec8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:13.318825 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585200dd-3255-4e04-8d00-4158c80b1ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:13.319215 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585200dd-3255-4e04-8d00-4158c80b1ec8 admin admin] PATH_INFO: `/v3/projects/646bce3324f44557841140de5f82e4ec` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:13.320357 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585200dd-3255-4e04-8d00-4158c80b1ec8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=646bce3324f44557841140de5f82e4ec)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:13.322641 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585200dd-3255-4e04-8d00-4158c80b1ec8 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:13.477794 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:13.479017 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-585200dd-3255-4e04-8d00-4158c80b1ec8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '646bce3324f44557841140de5f82e4ec', 'request_id': 'req-585200dd-3255-4e04-8d00-4158c80b1ec8'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:13.552633 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-585200dd-3255-4e04-8d00-4158c80b1ec8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '646bce3324f44557841140de5f82e4ec', 'request_id': 'req-585200dd-3255-4e04-8d00-4158c80b1ec8'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:13.560938 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-585200dd-3255-4e04-8d00-4158c80b1ec8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '646bce3324f44557841140de5f82e4ec', 'request_id': 'req-585200dd-3255-4e04-8d00-4158c80b1ec8'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:13.575849 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-585200dd-3255-4e04-8d00-4158c80b1ec8 admin admin] The token cache is being invalidate because project 646bce3324f44557841140de5f82e4ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:49:13.576286 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-585200dd-3255-4e04-8d00-4158c80b1ec8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:13.576988 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-585200dd-3255-4e04-8d00-4158c80b1ec8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:13.594267 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 908/1831] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:49:13 2022] DELETE /identity/v3/projects/646bce3324f44557841140de5f82e4ec => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:49:13.595218 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1d1b48-3943-4552-a5e6-48ebdf3d3165 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:13.595218 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1d1b48-3943-4552-a5e6-48ebdf3d3165 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:13.595218 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1d1b48-3943-4552-a5e6-48ebdf3d3165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:13.618005 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-fe1d1b48-3943-4552-a5e6-48ebdf3d3165 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:49:13.618005 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-fe1d1b48-3943-4552-a5e6-48ebdf3d3165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:13.691959 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 924/1832] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:49:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:49:13.700172 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a7648-8ca8-4f54-b4a6-ac24096b81dd None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:13.700520 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a7648-8ca8-4f54-b4a6-ac24096b81dd None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:13.712681 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a7648-8ca8-4f54-b4a6-ac24096b81dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:13.727749 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-778a7648-8ca8-4f54-b4a6-ac24096b81dd None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:49:13.729104 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-778a7648-8ca8-4f54-b4a6-ac24096b81dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:13.745008 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 909/1833] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:49:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:49:13.746396 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1d1b48-3943-4552-a5e6-48ebdf3d3165 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:13.758125 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1d1b48-3943-4552-a5e6-48ebdf3d3165 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:13.763287 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb1245-b93a-4f9d-9540-b6ded0da50ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:13.763901 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb1245-b93a-4f9d-9540-b6ded0da50ce admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:13.764177 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb1245-b93a-4f9d-9540-b6ded0da50ce admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:13.764448 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb1245-b93a-4f9d-9540-b6ded0da50ce admin admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:13.765124 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb1245-b93a-4f9d-9540-b6ded0da50ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:49:13.770008 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb1245-b93a-4f9d-9540-b6ded0da50ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:13.770008 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb1245-b93a-4f9d-9540-b6ded0da50ce admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:13.780709 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 925/1834] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:49:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:49:13.785421 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778a7648-8ca8-4f54-b4a6-ac24096b81dd None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:13.797347 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778a7648-8ca8-4f54-b4a6-ac24096b81dd None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:13.804221 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9381a112-3802-4bb6-a1d2-051eb9832b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:13.810467 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9381a112-3802-4bb6-a1d2-051eb9832b6d admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:13.810467 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9381a112-3802-4bb6-a1d2-051eb9832b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:13.810467 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9381a112-3802-4bb6-a1d2-051eb9832b6d admin admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:13.810467 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9381a112-3802-4bb6-a1d2-051eb9832b6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:13.810467 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9381a112-3802-4bb6-a1d2-051eb9832b6d admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:13.900200 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 910/1835] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:49:13 2022] POST /identity/v3/projects => generated 373 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:49:13.904444 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb1245-b93a-4f9d-9540-b6ded0da50ce admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:13.914608 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb1245-b93a-4f9d-9540-b6ded0da50ce admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:13.974308 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f383089-babb-4001-9551-c2e6dc2c7ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:13.974930 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f383089-babb-4001-9551-c2e6dc2c7ecf admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:13.975214 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f383089-babb-4001-9551-c2e6dc2c7ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:13.975578 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f383089-babb-4001-9551-c2e6dc2c7ecf admin admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:13.976393 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f383089-babb-4001-9551-c2e6dc2c7ecf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:13.978450 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f383089-babb-4001-9551-c2e6dc2c7ecf admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:14.089798 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 926/1836] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:49:13 2022] POST /identity/v3/users => generated 287 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:49:14.091061 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9381a112-3802-4bb6-a1d2-051eb9832b6d admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:14.103183 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9381a112-3802-4bb6-a1d2-051eb9832b6d admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:14.108488 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fa20d0-24e8-4ce8-9f10-d42a108cda22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:14.109284 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa20d0-24e8-4ce8-9f10-d42a108cda22 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:14.115465 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa20d0-24e8-4ce8-9f10-d42a108cda22 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:14.115465 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa20d0-24e8-4ce8-9f10-d42a108cda22 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:14.115465 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fa20d0-24e8-4ce8-9f10-d42a108cda22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:14.115465 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fa20d0-24e8-4ce8-9f10-d42a108cda22 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:14.157602 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 911/1837] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:49:14.161759 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f383089-babb-4001-9551-c2e6dc2c7ecf admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:14.178389 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f383089-babb-4001-9551-c2e6dc2c7ecf admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:14.194698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b318918-af4e-484e-a89b-73c23a568de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:14.195368 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b318918-af4e-484e-a89b-73c23a568de5 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:14.195641 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b318918-af4e-484e-a89b-73c23a568de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:14.195918 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b318918-af4e-484e-a89b-73c23a568de5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:14.196646 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b318918-af4e-484e-a89b-73c23a568de5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:14.198750 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b318918-af4e-484e-a89b-73c23a568de5 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:14.211498 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 927/1838] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:49:14.216105 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fa20d0-24e8-4ce8-9f10-d42a108cda22 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:14.227307 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fa20d0-24e8-4ce8-9f10-d42a108cda22 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:14.232183 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fbe857-6b08-49c8-9fdc-7f907a464137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:14.232879 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fbe857-6b08-49c8-9fdc-7f907a464137 admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:14.233920 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fbe857-6b08-49c8-9fdc-7f907a464137 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:14.234212 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fbe857-6b08-49c8-9fdc-7f907a464137 admin admin] PATH_INFO: `/v3/projects/7dbca51cb31642a6a965e1d7171d5f54/users/0a23eaf72e834ccebbec25dbddf4964e/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:14.254505 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fbe857-6b08-49c8-9fdc-7f907a464137 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dbca51cb31642a6a965e1d7171d5f54, user_id=0a23eaf72e834ccebbec25dbddf4964e, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:14.257952 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fbe857-6b08-49c8-9fdc-7f907a464137 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:14.341727 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 912/1839] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:49:14 2022] PUT /identity/v3/projects/7dbca51cb31642a6a965e1d7171d5f54/users/0a23eaf72e834ccebbec25dbddf4964e/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:49:14.352795 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d5f165-b792-4335-b5fc-7f74e18a3468 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:14.353245 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d5f165-b792-4335-b5fc-7f74e18a3468 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:14.353554 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d5f165-b792-4335-b5fc-7f74e18a3468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:14.390628 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-f3d5f165-b792-4335-b5fc-7f74e18a3468 None None] MFA Rules not processed for user `0a23eaf72e834ccebbec25dbddf4964e`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:49:14.392941 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-f3d5f165-b792-4335-b5fc-7f74e18a3468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:14.413448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fbe857-6b08-49c8-9fdc-7f907a464137 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:14.415295 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-99fbe857-6b08-49c8-9fdc-7f907a464137 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:14.442306 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:14.489084 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 928/1840] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:49:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:49:14.493723 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d5f165-b792-4335-b5fc-7f74e18a3468 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:14.500436 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fbe857-6b08-49c8-9fdc-7f907a464137 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:14.505685 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d1d64-7aef-4bd6-8949-fac32695e14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:14.506390 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d1d64-7aef-4bd6-8949-fac32695e14e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:14.506682 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d1d64-7aef-4bd6-8949-fac32695e14e admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:14.506956 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d1d64-7aef-4bd6-8949-fac32695e14e admin admin] PATH_INFO: `/v3/users/ce59a8ef6c3640da8218defd5fbf2a76` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:14.507152 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d5f165-b792-4335-b5fc-7f74e18a3468 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:14.508095 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d1d64-7aef-4bd6-8949-fac32695e14e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce59a8ef6c3640da8218defd5fbf2a76)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:14.510052 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d1d64-7aef-4bd6-8949-fac32695e14e admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:14.512452 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d90ac4-fb11-4b4c-95ac-52d470e84463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:14.514885 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d90ac4-fb11-4b4c-95ac-52d470e84463 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:14.515585 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d90ac4-fb11-4b4c-95ac-52d470e84463 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:14.515926 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d90ac4-fb11-4b4c-95ac-52d470e84463 admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:14.517482 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d90ac4-fb11-4b4c-95ac-52d470e84463 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:14.518773 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d90ac4-fb11-4b4c-95ac-52d470e84463 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:14.681759 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 929/1841] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:49:14 2022] POST /identity/v3/projects => generated 373 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:49:14.682891 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d90ac4-fb11-4b4c-95ac-52d470e84463 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:14.754459 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d90ac4-fb11-4b4c-95ac-52d470e84463 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:14.761377 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa646fb-22ee-425a-8ede-5a09ecfca503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:14.762117 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa646fb-22ee-425a-8ede-5a09ecfca503 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:14.762399 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa646fb-22ee-425a-8ede-5a09ecfca503 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:14.762671 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa646fb-22ee-425a-8ede-5a09ecfca503 admin admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:14.763306 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa646fb-22ee-425a-8ede-5a09ecfca503 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:14.765271 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa646fb-22ee-425a-8ede-5a09ecfca503 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:14.834949 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-139d1d64-7aef-4bd6-8949-fac32695e14e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce59a8ef6c3640da8218defd5fbf2a76'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:14.855986 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-139d1d64-7aef-4bd6-8949-fac32695e14e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce59a8ef6c3640da8218defd5fbf2a76'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:14.882158 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-139d1d64-7aef-4bd6-8949-fac32695e14e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce59a8ef6c3640da8218defd5fbf2a76'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:14.884894 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=85,keystone:INSERT=3,keystone:DELETE=12 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:14.929271 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 930/1842] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:49:14 2022] POST /identity/v3/users => generated 293 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:49:14.933357 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa646fb-22ee-425a-8ede-5a09ecfca503 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:14.955314 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa646fb-22ee-425a-8ede-5a09ecfca503 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:14.957654 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad93cf60-fb29-4b2a-bcb2-7ff92bdcde5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:14.958261 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad93cf60-fb29-4b2a-bcb2-7ff92bdcde5f admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:14.958546 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad93cf60-fb29-4b2a-bcb2-7ff92bdcde5f admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:14.958822 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad93cf60-fb29-4b2a-bcb2-7ff92bdcde5f admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:14.959469 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad93cf60-fb29-4b2a-bcb2-7ff92bdcde5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:14.961098 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad93cf60-fb29-4b2a-bcb2-7ff92bdcde5f admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:14.972144 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 931/1843] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:49:14.975794 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-139d1d64-7aef-4bd6-8949-fac32695e14e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce59a8ef6c3640da8218defd5fbf2a76'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:14.976404 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad93cf60-fb29-4b2a-bcb2-7ff92bdcde5f admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:14.977124 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-139d1d64-7aef-4bd6-8949-fac32695e14e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce59a8ef6c3640da8218defd5fbf2a76'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:14.985032 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 913/1844] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:49:14 2022] DELETE /identity/v3/users/ce59a8ef6c3640da8218defd5fbf2a76 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:49:14.986832 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad93cf60-fb29-4b2a-bcb2-7ff92bdcde5f admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:14.989507 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93852139-17ca-4572-90e1-b9c6892c2389 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:14.990224 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93852139-17ca-4572-90e1-b9c6892c2389 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:14.991190 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93852139-17ca-4572-90e1-b9c6892c2389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:15.029896 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-93852139-17ca-4572-90e1-b9c6892c2389 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:49:15.031266 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-93852139-17ca-4572-90e1-b9c6892c2389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:15.059904 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f2723-d92d-4895-9628-2dcde7371468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:15.060663 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f2723-d92d-4895-9628-2dcde7371468 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:15.060885 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f2723-d92d-4895-9628-2dcde7371468 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:15.061162 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f2723-d92d-4895-9628-2dcde7371468 admin admin] PATH_INFO: `/v3/projects/5b7434a3f45f4bfea9d699069b12f988/users/8ffffc6cb9484380a90127ef5a3e938a/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:15.082900 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7f2723-d92d-4895-9628-2dcde7371468 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b7434a3f45f4bfea9d699069b12f988, user_id=8ffffc6cb9484380a90127ef5a3e938a, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:15.087956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7f2723-d92d-4895-9628-2dcde7371468 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:15.107578 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 914/1845] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:49:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:49:15.119097 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93852139-17ca-4572-90e1-b9c6892c2389 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:15.120495 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-93852139-17ca-4572-90e1-b9c6892c2389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:15.191850 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93852139-17ca-4572-90e1-b9c6892c2389 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:15.197039 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 932/1846] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:49:14 2022] PUT /identity/v3/projects/5b7434a3f45f4bfea9d699069b12f988/users/8ffffc6cb9484380a90127ef5a3e938a/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:49:15.199275 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f2723-d92d-4895-9628-2dcde7371468 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:15.200477 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-3b7f2723-d92d-4895-9628-2dcde7371468 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:15.223949 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:15.254351 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97becd44-3978-48b4-94d1-6fd81defa011 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:15.255004 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97becd44-3978-48b4-94d1-6fd81defa011 service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:15.255268 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97becd44-3978-48b4-94d1-6fd81defa011 service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:15.255492 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97becd44-3978-48b4-94d1-6fd81defa011 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:15.283895 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f2723-d92d-4895-9628-2dcde7371468 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:15.297551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf283bc-f17f-4d3a-ada5-ddf011e83285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:15.297551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf283bc-f17f-4d3a-ada5-ddf011e83285 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:15.297551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf283bc-f17f-4d3a-ada5-ddf011e83285 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:15.297551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf283bc-f17f-4d3a-ada5-ddf011e83285 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:15.300292 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf283bc-f17f-4d3a-ada5-ddf011e83285 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:15.303256 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf283bc-f17f-4d3a-ada5-ddf011e83285 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:15.328693 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 933/1847] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:49:15 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:49:15.333140 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf283bc-f17f-4d3a-ada5-ddf011e83285 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:15.343608 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf283bc-f17f-4d3a-ada5-ddf011e83285 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:15.348959 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81131f27-5e72-4064-8c27-98c47b4a0147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:15.349629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81131f27-5e72-4064-8c27-98c47b4a0147 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:15.349930 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81131f27-5e72-4064-8c27-98c47b4a0147 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:15.350189 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81131f27-5e72-4064-8c27-98c47b4a0147 admin admin] PATH_INFO: `/v3/projects/5b7434a3f45f4bfea9d699069b12f988/users/8ffffc6cb9484380a90127ef5a3e938a/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:15.352229 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81131f27-5e72-4064-8c27-98c47b4a0147 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b7434a3f45f4bfea9d699069b12f988, user_id=8ffffc6cb9484380a90127ef5a3e938a, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:15.355393 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81131f27-5e72-4064-8c27-98c47b4a0147 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:15.366295 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97becd44-3978-48b4-94d1-6fd81defa011 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:15.368070 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97becd44-3978-48b4-94d1-6fd81defa011 service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:15.371889 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 934/1848] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:49:15 2022] PUT /identity/v3/projects/5b7434a3f45f4bfea9d699069b12f988/users/8ffffc6cb9484380a90127ef5a3e938a/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:49:15.372560 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:15.380823 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 915/1849] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:49:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:49:15.382681 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a7197d-a8a8-4f4e-9bb7-4067d1a5da97 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:15.382978 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a7197d-a8a8-4f4e-9bb7-4067d1a5da97 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:15.383304 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a7197d-a8a8-4f4e-9bb7-4067d1a5da97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:15.416607 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-66a7197d-a8a8-4f4e-9bb7-4067d1a5da97 None None] MFA Rules not processed for user `8ffffc6cb9484380a90127ef5a3e938a`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:49:15.417877 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-66a7197d-a8a8-4f4e-9bb7-4067d1a5da97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:15.552767 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 935/1850] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:49:15 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:49:15.688183 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97becd44-3978-48b4-94d1-6fd81defa011 service neutron] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:15.693601 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-97becd44-3978-48b4-94d1-6fd81defa011 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:15.754423 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97becd44-3978-48b4-94d1-6fd81defa011 service neutron] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:15.812164 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebdb96e-8c90-4791-87d9-d940f946dc47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:15.812984 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebdb96e-8c90-4791-87d9-d940f946dc47 service cinder] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:15.813408 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebdb96e-8c90-4791-87d9-d940f946dc47 service cinder] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:15.813831 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebdb96e-8c90-4791-87d9-d940f946dc47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:15.824213 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebdb96e-8c90-4791-87d9-d940f946dc47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:15.825805 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebdb96e-8c90-4791-87d9-d940f946dc47 service cinder] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:15.836416 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 916/1851] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:49:15 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:49:16.020262 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a7197d-a8a8-4f4e-9bb7-4067d1a5da97 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:16.020262 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-66a7197d-a8a8-4f4e-9bb7-4067d1a5da97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:16.128211 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a7197d-a8a8-4f4e-9bb7-4067d1a5da97 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:16.134060 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc6e184-6db0-4163-b0cd-f27d882f08b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:16.134853 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc6e184-6db0-4163-b0cd-f27d882f08b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:16.135131 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc6e184-6db0-4163-b0cd-f27d882f08b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:16.135406 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc6e184-6db0-4163-b0cd-f27d882f08b5 admin admin] PATH_INFO: `/v3/projects/69b6daa432644afc90047d9cb7164c4c` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:16.136557 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc6e184-6db0-4163-b0cd-f27d882f08b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69b6daa432644afc90047d9cb7164c4c)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:16.138556 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc6e184-6db0-4163-b0cd-f27d882f08b5 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:16.180429 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebdb96e-8c90-4791-87d9-d940f946dc47 service cinder] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:16.193695 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebdb96e-8c90-4791-87d9-d940f946dc47 service cinder] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:16.197517 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4748a103-3685-4b65-911d-871a942da006 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:16.198318 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4748a103-3685-4b65-911d-871a942da006 service cinder] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:16.198696 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4748a103-3685-4b65-911d-871a942da006 service cinder] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:16.198874 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4748a103-3685-4b65-911d-871a942da006 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:16.201503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-4748a103-3685-4b65-911d-871a942da006 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:16.268616 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:16.285909 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4748a103-3685-4b65-911d-871a942da006 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:16.289055 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4748a103-3685-4b65-911d-871a942da006 service cinder] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:16.308272 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-cfc6e184-6db0-4163-b0cd-f27d882f08b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69b6daa432644afc90047d9cb7164c4c', 'request_id': 'req-cfc6e184-6db0-4163-b0cd-f27d882f08b5'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:16.310446 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 917/1852] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:49:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:49:16.327242 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-cfc6e184-6db0-4163-b0cd-f27d882f08b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69b6daa432644afc90047d9cb7164c4c', 'request_id': 'req-cfc6e184-6db0-4163-b0cd-f27d882f08b5'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:16.337143 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-cfc6e184-6db0-4163-b0cd-f27d882f08b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69b6daa432644afc90047d9cb7164c4c', 'request_id': 'req-cfc6e184-6db0-4163-b0cd-f27d882f08b5'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:16.424882 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-cfc6e184-6db0-4163-b0cd-f27d882f08b5 admin admin] The token cache is being invalidate because project 69b6daa432644afc90047d9cb7164c4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:49:16.425171 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-cfc6e184-6db0-4163-b0cd-f27d882f08b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:16.425813 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-cfc6e184-6db0-4163-b0cd-f27d882f08b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:16.427307 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 936/1853] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:49:16 2022] DELETE /identity/v3/projects/69b6daa432644afc90047d9cb7164c4c => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:49:16.431360 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4748a103-3685-4b65-911d-871a942da006 service cinder] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:16.443609 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4748a103-3685-4b65-911d-871a942da006 service cinder] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:16.504496 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:16.511965 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681370a2-6313-4183-8ac7-27b4f53f6c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:16.513266 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681370a2-6313-4183-8ac7-27b4f53f6c65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:16.513602 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681370a2-6313-4183-8ac7-27b4f53f6c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:16.513883 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681370a2-6313-4183-8ac7-27b4f53f6c65 admin admin] PATH_INFO: `/v3/users/45610afe1e894ccf9c84eb03ee4a4d1f` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:16.515302 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681370a2-6313-4183-8ac7-27b4f53f6c65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45610afe1e894ccf9c84eb03ee4a4d1f)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:16.518084 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681370a2-6313-4183-8ac7-27b4f53f6c65 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:16.741281 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-681370a2-6313-4183-8ac7-27b4f53f6c65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45610afe1e894ccf9c84eb03ee4a4d1f'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:16.754546 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-681370a2-6313-4183-8ac7-27b4f53f6c65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45610afe1e894ccf9c84eb03ee4a4d1f'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:16.761260 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-681370a2-6313-4183-8ac7-27b4f53f6c65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45610afe1e894ccf9c84eb03ee4a4d1f'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:16.864646 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-681370a2-6313-4183-8ac7-27b4f53f6c65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45610afe1e894ccf9c84eb03ee4a4d1f'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:16.865547 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-681370a2-6313-4183-8ac7-27b4f53f6c65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45610afe1e894ccf9c84eb03ee4a4d1f'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:16.872768 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 918/1854] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:49:16 2022] DELETE /identity/v3/users/45610afe1e894ccf9c84eb03ee4a4d1f => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:49:17.358824 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc6e184-6db0-4163-b0cd-f27d882f08b5 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:17.367683 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-cfc6e184-6db0-4163-b0cd-f27d882f08b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:17.375700 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc6e184-6db0-4163-b0cd-f27d882f08b5 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:17.462347 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c746b91-146d-425a-979c-3ba732edb4da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:17.466324 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c746b91-146d-425a-979c-3ba732edb4da admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:17.466324 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c746b91-146d-425a-979c-3ba732edb4da admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:17.466324 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c746b91-146d-425a-979c-3ba732edb4da admin admin] PATH_INFO: `/v3/projects/5fb8b4901ec747829f20390a6441f140` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:17.466324 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c746b91-146d-425a-979c-3ba732edb4da admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fb8b4901ec747829f20390a6441f140)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:17.470983 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c746b91-146d-425a-979c-3ba732edb4da admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:17.669642 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-1c746b91-146d-425a-979c-3ba732edb4da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fb8b4901ec747829f20390a6441f140', 'request_id': 'req-1c746b91-146d-425a-979c-3ba732edb4da'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:17.679263 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-1c746b91-146d-425a-979c-3ba732edb4da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fb8b4901ec747829f20390a6441f140', 'request_id': 'req-1c746b91-146d-425a-979c-3ba732edb4da'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:17.686370 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-1c746b91-146d-425a-979c-3ba732edb4da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fb8b4901ec747829f20390a6441f140', 'request_id': 'req-1c746b91-146d-425a-979c-3ba732edb4da'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:17.805066 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-1c746b91-146d-425a-979c-3ba732edb4da admin admin] The token cache is being invalidate because project 5fb8b4901ec747829f20390a6441f140 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:49:17.805590 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-1c746b91-146d-425a-979c-3ba732edb4da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:17.806453 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-1c746b91-146d-425a-979c-3ba732edb4da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:17.808283 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 937/1855] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:49:17 2022] DELETE /identity/v3/projects/5fb8b4901ec747829f20390a6441f140 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:49:17.816536 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2efc8-a151-4e4b-ba66-7e33e92e7ad1 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:17.817648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2efc8-a151-4e4b-ba66-7e33e92e7ad1 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:17.818117 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2efc8-a151-4e4b-ba66-7e33e92e7ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:17.841295 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-4dd2efc8-a151-4e4b-ba66-7e33e92e7ad1 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:49:17.842776 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-4dd2efc8-a151-4e4b-ba66-7e33e92e7ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:17.946545 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 919/1856] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:49:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:49:17.955187 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af4a3a-6baf-4663-b7e5-3ca499e112b9 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:17.957368 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af4a3a-6baf-4663-b7e5-3ca499e112b9 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:17.957368 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af4a3a-6baf-4663-b7e5-3ca499e112b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:17.984542 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-46af4a3a-6baf-4663-b7e5-3ca499e112b9 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:49:17.984542 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-46af4a3a-6baf-4663-b7e5-3ca499e112b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:17.993625 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 938/1857] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:49:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:49:17.998343 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd2efc8-a151-4e4b-ba66-7e33e92e7ad1 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:18.009647 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd2efc8-a151-4e4b-ba66-7e33e92e7ad1 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:18.014317 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2aa4c0-2afd-484d-9ac8-1515a838ab28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:18.024477 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2aa4c0-2afd-484d-9ac8-1515a838ab28 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:18.024477 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2aa4c0-2afd-484d-9ac8-1515a838ab28 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:18.024477 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2aa4c0-2afd-484d-9ac8-1515a838ab28 admin admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:18.024477 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2aa4c0-2afd-484d-9ac8-1515a838ab28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:49:18.024477 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2aa4c0-2afd-484d-9ac8-1515a838ab28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:18.024477 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2aa4c0-2afd-484d-9ac8-1515a838ab28 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:18.031576 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 920/1858] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:49:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:49:18.037611 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af4a3a-6baf-4663-b7e5-3ca499e112b9 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:18.047503 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af4a3a-6baf-4663-b7e5-3ca499e112b9 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:18.055716 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3178af89-a264-4895-9b67-41d7ef7bee33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:18.056370 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178af89-a264-4895-9b67-41d7ef7bee33 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:18.056650 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178af89-a264-4895-9b67-41d7ef7bee33 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:18.056953 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178af89-a264-4895-9b67-41d7ef7bee33 admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:18.057595 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3178af89-a264-4895-9b67-41d7ef7bee33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:18.060057 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3178af89-a264-4895-9b67-41d7ef7bee33 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:18.151732 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 939/1859] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:49:18 2022] POST /identity/v3/projects => generated 375 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:49:18.156115 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2aa4c0-2afd-484d-9ac8-1515a838ab28 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:18.166859 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2aa4c0-2afd-484d-9ac8-1515a838ab28 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:18.231870 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e2e893-c21a-476a-b3a8-829732b19637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:18.232673 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2e893-c21a-476a-b3a8-829732b19637 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:18.233086 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2e893-c21a-476a-b3a8-829732b19637 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:18.233527 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2e893-c21a-476a-b3a8-829732b19637 admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:18.234293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e2e893-c21a-476a-b3a8-829732b19637 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:18.236943 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e2e893-c21a-476a-b3a8-829732b19637 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:18.251223 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:18.389029 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 921/1860] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:49:18 2022] POST /identity/v3/users => generated 288 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:49:18.390633 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3178af89-a264-4895-9b67-41d7ef7bee33 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:18.401693 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3178af89-a264-4895-9b67-41d7ef7bee33 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:18.406288 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f554ac23-111b-4d7b-b68f-48a3dd3052f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:18.406890 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554ac23-111b-4d7b-b68f-48a3dd3052f4 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:18.407204 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554ac23-111b-4d7b-b68f-48a3dd3052f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:18.408051 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554ac23-111b-4d7b-b68f-48a3dd3052f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:18.408808 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f554ac23-111b-4d7b-b68f-48a3dd3052f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:18.410425 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f554ac23-111b-4d7b-b68f-48a3dd3052f4 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:18.421885 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 940/1861] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:49:18.426120 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e2e893-c21a-476a-b3a8-829732b19637 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:18.436441 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e2e893-c21a-476a-b3a8-829732b19637 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:18.441730 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d151595-665b-4fda-8a0b-005df3431e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:18.442199 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d151595-665b-4fda-8a0b-005df3431e1c admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:18.442845 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d151595-665b-4fda-8a0b-005df3431e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:18.443370 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d151595-665b-4fda-8a0b-005df3431e1c admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:18.444532 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d151595-665b-4fda-8a0b-005df3431e1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:18.446273 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d151595-665b-4fda-8a0b-005df3431e1c admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:18.460014 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 922/1862] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:49:18.464587 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f554ac23-111b-4d7b-b68f-48a3dd3052f4 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:18.472536 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:18.476367 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f554ac23-111b-4d7b-b68f-48a3dd3052f4 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:18.482139 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31e02f-b08e-4e09-b3d0-be09a8f37014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:18.482514 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31e02f-b08e-4e09-b3d0-be09a8f37014 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:18.482795 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31e02f-b08e-4e09-b3d0-be09a8f37014 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:18.483099 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31e02f-b08e-4e09-b3d0-be09a8f37014 admin admin] PATH_INFO: `/v3/projects/a9eeacff0e01442c9a739b43590284d0/users/d4d4dc994f9744ab990866ae8def199d/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:18.505758 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b31e02f-b08e-4e09-b3d0-be09a8f37014 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9eeacff0e01442c9a739b43590284d0, user_id=d4d4dc994f9744ab990866ae8def199d, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:18.508860 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b31e02f-b08e-4e09-b3d0-be09a8f37014 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:18.613701 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 941/1863] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:49:18 2022] PUT /identity/v3/projects/a9eeacff0e01442c9a739b43590284d0/users/d4d4dc994f9744ab990866ae8def199d/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:49:18.626473 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37556aa-3c2a-419f-9181-c1e0ca4c1589 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:18.626776 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37556aa-3c2a-419f-9181-c1e0ca4c1589 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:18.627058 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37556aa-3c2a-419f-9181-c1e0ca4c1589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:18.657217 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-f37556aa-3c2a-419f-9181-c1e0ca4c1589 None None] MFA Rules not processed for user `d4d4dc994f9744ab990866ae8def199d`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:49:18.659351 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-f37556aa-3c2a-419f-9181-c1e0ca4c1589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:18.735181 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 923/1864] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:49:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:49:18.754099 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31e02f-b08e-4e09-b3d0-be09a8f37014 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:18.755985 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-6b31e02f-b08e-4e09-b3d0-be09a8f37014 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:18.818221 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31e02f-b08e-4e09-b3d0-be09a8f37014 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:18.822215 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d759514c-c73c-44c1-8f29-00d8fb1e8932 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:18.822945 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d759514c-c73c-44c1-8f29-00d8fb1e8932 service cinder] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:18.823323 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d759514c-c73c-44c1-8f29-00d8fb1e8932 service cinder] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:18.823548 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d759514c-c73c-44c1-8f29-00d8fb1e8932 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:18.834126 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d759514c-c73c-44c1-8f29-00d8fb1e8932 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:18.835566 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d759514c-c73c-44c1-8f29-00d8fb1e8932 service cinder] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:18.846175 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 942/1865] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:49:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:49:28.732049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:INSERT=2,keystone:SELECT=39 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:28.841137 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=34,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:39.917593 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37556aa-3c2a-419f-9181-c1e0ca4c1589 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:39.919125 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-f37556aa-3c2a-419f-9181-c1e0ca4c1589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:39.988177 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37556aa-3c2a-419f-9181-c1e0ca4c1589 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:39.994271 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3cce4e-7718-4b3d-a8e0-f9877b024735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:39.995488 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3cce4e-7718-4b3d-a8e0-f9877b024735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:39.995842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3cce4e-7718-4b3d-a8e0-f9877b024735 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:39.996087 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3cce4e-7718-4b3d-a8e0-f9877b024735 admin admin] PATH_INFO: `/v3/users/88497a4e7f5c4f73a6f1627a5c8e9d66` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:39.997229 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3cce4e-7718-4b3d-a8e0-f9877b024735 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88497a4e7f5c4f73a6f1627a5c8e9d66)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:39.999192 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3cce4e-7718-4b3d-a8e0-f9877b024735 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:40.269890 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-3c3cce4e-7718-4b3d-a8e0-f9877b024735 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88497a4e7f5c4f73a6f1627a5c8e9d66'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:40.282454 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-3c3cce4e-7718-4b3d-a8e0-f9877b024735 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88497a4e7f5c4f73a6f1627a5c8e9d66'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:40.289682 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-3c3cce4e-7718-4b3d-a8e0-f9877b024735 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88497a4e7f5c4f73a6f1627a5c8e9d66'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:40.391777 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-3c3cce4e-7718-4b3d-a8e0-f9877b024735 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88497a4e7f5c4f73a6f1627a5c8e9d66'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:40.392451 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-3c3cce4e-7718-4b3d-a8e0-f9877b024735 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88497a4e7f5c4f73a6f1627a5c8e9d66'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:40.399525 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 924/1866] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:49:39 2022] DELETE /identity/v3/users/88497a4e7f5c4f73a6f1627a5c8e9d66 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:49:40.404107 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab8f3cd-f53c-47de-919b-8aacd884acff None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:40.404619 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab8f3cd-f53c-47de-919b-8aacd884acff None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:40.404880 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab8f3cd-f53c-47de-919b-8aacd884acff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:40.427569 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-dab8f3cd-f53c-47de-919b-8aacd884acff None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:49:40.428884 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-dab8f3cd-f53c-47de-919b-8aacd884acff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:40.497345 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 943/1867] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:49:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:49:40.510293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3cce4e-7718-4b3d-a8e0-f9877b024735 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:40.510293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-3c3cce4e-7718-4b3d-a8e0-f9877b024735 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:40.588042 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3cce4e-7718-4b3d-a8e0-f9877b024735 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:40.592215 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ee2b61-4b01-43df-80d2-a1bcbb6fc6c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:40.593137 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee2b61-4b01-43df-80d2-a1bcbb6fc6c3 service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:40.593522 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee2b61-4b01-43df-80d2-a1bcbb6fc6c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:40.594075 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee2b61-4b01-43df-80d2-a1bcbb6fc6c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:40.606065 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ee2b61-4b01-43df-80d2-a1bcbb6fc6c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:40.607996 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ee2b61-4b01-43df-80d2-a1bcbb6fc6c3 service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:40.621943 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 925/1868] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:49:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:49:41.167863 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab8f3cd-f53c-47de-919b-8aacd884acff None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:41.167863 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-dab8f3cd-f53c-47de-919b-8aacd884acff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:41.181198 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab8f3cd-f53c-47de-919b-8aacd884acff None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:41.186173 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9352a624-67ef-4dfa-ae93-dfe2c2d19b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:41.186814 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352a624-67ef-4dfa-ae93-dfe2c2d19b93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:41.187099 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352a624-67ef-4dfa-ae93-dfe2c2d19b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:41.187381 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352a624-67ef-4dfa-ae93-dfe2c2d19b93 admin admin] PATH_INFO: `/v3/projects/d1ba01bfd210473cb1eef75894b28b32` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:41.188666 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9352a624-67ef-4dfa-ae93-dfe2c2d19b93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1ba01bfd210473cb1eef75894b28b32)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:41.191364 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9352a624-67ef-4dfa-ae93-dfe2c2d19b93 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:41.324773 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-9352a624-67ef-4dfa-ae93-dfe2c2d19b93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1ba01bfd210473cb1eef75894b28b32', 'request_id': 'req-9352a624-67ef-4dfa-ae93-dfe2c2d19b93'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:41.330747 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-9352a624-67ef-4dfa-ae93-dfe2c2d19b93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1ba01bfd210473cb1eef75894b28b32', 'request_id': 'req-9352a624-67ef-4dfa-ae93-dfe2c2d19b93'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:41.337156 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-9352a624-67ef-4dfa-ae93-dfe2c2d19b93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1ba01bfd210473cb1eef75894b28b32', 'request_id': 'req-9352a624-67ef-4dfa-ae93-dfe2c2d19b93'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:41.392417 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-9352a624-67ef-4dfa-ae93-dfe2c2d19b93 admin admin] The token cache is being invalidate because project d1ba01bfd210473cb1eef75894b28b32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:49:41.397785 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-9352a624-67ef-4dfa-ae93-dfe2c2d19b93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:41.397785 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-9352a624-67ef-4dfa-ae93-dfe2c2d19b93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:41.397785 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 944/1869] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:49:41 2022] DELETE /identity/v3/projects/d1ba01bfd210473cb1eef75894b28b32 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:49:41.400639 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ee2b61-4b01-43df-80d2-a1bcbb6fc6c3 service neutron] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:41.419872 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ee2b61-4b01-43df-80d2-a1bcbb6fc6c3 service neutron] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:41.432651 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:41.473259 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfe9de5-e4a1-404b-b53a-b6f0cdc41eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:41.473933 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe9de5-e4a1-404b-b53a-b6f0cdc41eb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:41.474336 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe9de5-e4a1-404b-b53a-b6f0cdc41eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:41.474617 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe9de5-e4a1-404b-b53a-b6f0cdc41eb2 admin admin] PATH_INFO: `/v3/users/2e00a82541ae4484b05fa765df939b33` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:41.475779 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfe9de5-e4a1-404b-b53a-b6f0cdc41eb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e00a82541ae4484b05fa765df939b33)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:41.477800 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfe9de5-e4a1-404b-b53a-b6f0cdc41eb2 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:41.718700 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-1cfe9de5-e4a1-404b-b53a-b6f0cdc41eb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e00a82541ae4484b05fa765df939b33'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:41.733136 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-1cfe9de5-e4a1-404b-b53a-b6f0cdc41eb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e00a82541ae4484b05fa765df939b33'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:41.741512 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-1cfe9de5-e4a1-404b-b53a-b6f0cdc41eb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e00a82541ae4484b05fa765df939b33'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:41.825942 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-1cfe9de5-e4a1-404b-b53a-b6f0cdc41eb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e00a82541ae4484b05fa765df939b33'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:41.826595 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-1cfe9de5-e4a1-404b-b53a-b6f0cdc41eb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e00a82541ae4484b05fa765df939b33'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:41.833937 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 926/1870] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:49:41 2022] DELETE /identity/v3/users/2e00a82541ae4484b05fa765df939b33 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:49:42.277805 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9352a624-67ef-4dfa-ae93-dfe2c2d19b93 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:42.279375 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-9352a624-67ef-4dfa-ae93-dfe2c2d19b93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:42.293912 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9352a624-67ef-4dfa-ae93-dfe2c2d19b93 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:42.354135 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce54930d-1e2c-405e-806e-f89a5270cf59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:42.354796 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54930d-1e2c-405e-806e-f89a5270cf59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:42.355078 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54930d-1e2c-405e-806e-f89a5270cf59 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:42.355353 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54930d-1e2c-405e-806e-f89a5270cf59 admin admin] PATH_INFO: `/v3/projects/eb4d3967835b4c97bb12da77d1b5946f` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:42.356472 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce54930d-1e2c-405e-806e-f89a5270cf59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb4d3967835b4c97bb12da77d1b5946f)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:42.358481 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce54930d-1e2c-405e-806e-f89a5270cf59 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:42.438607 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:42.455094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ce54930d-1e2c-405e-806e-f89a5270cf59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb4d3967835b4c97bb12da77d1b5946f', 'request_id': 'req-ce54930d-1e2c-405e-806e-f89a5270cf59'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:42.460941 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ce54930d-1e2c-405e-806e-f89a5270cf59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb4d3967835b4c97bb12da77d1b5946f', 'request_id': 'req-ce54930d-1e2c-405e-806e-f89a5270cf59'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:42.467056 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ce54930d-1e2c-405e-806e-f89a5270cf59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb4d3967835b4c97bb12da77d1b5946f', 'request_id': 'req-ce54930d-1e2c-405e-806e-f89a5270cf59'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:42.555910 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ce54930d-1e2c-405e-806e-f89a5270cf59 admin admin] The token cache is being invalidate because project eb4d3967835b4c97bb12da77d1b5946f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:49:42.556201 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ce54930d-1e2c-405e-806e-f89a5270cf59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:42.556894 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ce54930d-1e2c-405e-806e-f89a5270cf59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:42.558401 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 945/1871] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:49:42 2022] DELETE /identity/v3/projects/eb4d3967835b4c97bb12da77d1b5946f => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:49:42.563276 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff100f79-b5a6-430d-aff4-c8f70e8c1608 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:42.563745 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff100f79-b5a6-430d-aff4-c8f70e8c1608 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:42.563867 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff100f79-b5a6-430d-aff4-c8f70e8c1608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:42.585339 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-ff100f79-b5a6-430d-aff4-c8f70e8c1608 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:49:42.586568 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-ff100f79-b5a6-430d-aff4-c8f70e8c1608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:42.647338 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 927/1872] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:49:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:49:42.655462 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f1ac7-47c9-4783-ae05-2a709a2fb548 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:42.655756 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f1ac7-47c9-4783-ae05-2a709a2fb548 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:42.656037 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f1ac7-47c9-4783-ae05-2a709a2fb548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:42.677793 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-932f1ac7-47c9-4783-ae05-2a709a2fb548 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:49:42.679019 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-932f1ac7-47c9-4783-ae05-2a709a2fb548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:42.687087 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 946/1873] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:49:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:49:42.692035 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff100f79-b5a6-430d-aff4-c8f70e8c1608 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:42.701840 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff100f79-b5a6-430d-aff4-c8f70e8c1608 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:42.706830 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e6bff-788d-47cd-8839-fe0cd3e2b19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:42.707396 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e6bff-788d-47cd-8839-fe0cd3e2b19a admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:42.707677 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e6bff-788d-47cd-8839-fe0cd3e2b19a admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:42.707953 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e6bff-788d-47cd-8839-fe0cd3e2b19a admin admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:42.708597 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722e6bff-788d-47cd-8839-fe0cd3e2b19a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:49:42.708958 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722e6bff-788d-47cd-8839-fe0cd3e2b19a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:42.710563 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722e6bff-788d-47cd-8839-fe0cd3e2b19a admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:42.722751 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 928/1874] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:49:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:49:42.727136 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932f1ac7-47c9-4783-ae05-2a709a2fb548 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:42.736616 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932f1ac7-47c9-4783-ae05-2a709a2fb548 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:42.747601 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d48d57f-21c6-4c6d-9718-339b8338db7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:42.748654 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d48d57f-21c6-4c6d-9718-339b8338db7c admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:42.749186 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d48d57f-21c6-4c6d-9718-339b8338db7c admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:42.749811 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d48d57f-21c6-4c6d-9718-339b8338db7c admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:42.750900 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d48d57f-21c6-4c6d-9718-339b8338db7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:42.754473 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d48d57f-21c6-4c6d-9718-339b8338db7c admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:42.849113 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 947/1875] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:49:42 2022] POST /identity/v3/projects => generated 409 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:49:42.853616 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e6bff-788d-47cd-8839-fe0cd3e2b19a admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:42.868592 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e6bff-788d-47cd-8839-fe0cd3e2b19a admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:42.904894 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:42.936043 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dd9114-c5ae-4fb1-aceb-3ee8b878be22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:42.936644 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd9114-c5ae-4fb1-aceb-3ee8b878be22 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:42.936955 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd9114-c5ae-4fb1-aceb-3ee8b878be22 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:42.937232 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd9114-c5ae-4fb1-aceb-3ee8b878be22 admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:42.937871 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dd9114-c5ae-4fb1-aceb-3ee8b878be22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:42.939792 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dd9114-c5ae-4fb1-aceb-3ee8b878be22 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:43.116609 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 929/1876] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:49:42 2022] POST /identity/v3/users => generated 305 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:49:43.120235 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d48d57f-21c6-4c6d-9718-339b8338db7c admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:43.131230 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d48d57f-21c6-4c6d-9718-339b8338db7c admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:43.135907 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6732e91-35dd-48eb-8e4b-baa49f8a5d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:43.136516 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6732e91-35dd-48eb-8e4b-baa49f8a5d45 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:43.136828 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6732e91-35dd-48eb-8e4b-baa49f8a5d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:43.137104 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6732e91-35dd-48eb-8e4b-baa49f8a5d45 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:43.137748 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6732e91-35dd-48eb-8e4b-baa49f8a5d45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:43.139348 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6732e91-35dd-48eb-8e4b-baa49f8a5d45 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:43.151304 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 948/1877] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:49:43.155786 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dd9114-c5ae-4fb1-aceb-3ee8b878be22 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:43.167401 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dd9114-c5ae-4fb1-aceb-3ee8b878be22 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:43.172776 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1999e6b-2259-4b48-ab1d-9c0ddf70fdc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:43.173651 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1999e6b-2259-4b48-ab1d-9c0ddf70fdc9 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:43.173916 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1999e6b-2259-4b48-ab1d-9c0ddf70fdc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:43.174340 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1999e6b-2259-4b48-ab1d-9c0ddf70fdc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:43.175181 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1999e6b-2259-4b48-ab1d-9c0ddf70fdc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:43.177342 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1999e6b-2259-4b48-ab1d-9c0ddf70fdc9 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:43.188822 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 930/1878] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:49:43.193335 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6732e91-35dd-48eb-8e4b-baa49f8a5d45 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:43.203081 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6732e91-35dd-48eb-8e4b-baa49f8a5d45 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:43.210624 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2feb9d-5c02-47c4-992f-dd7c69549ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:43.210624 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2feb9d-5c02-47c4-992f-dd7c69549ffe admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:43.210624 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2feb9d-5c02-47c4-992f-dd7c69549ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:43.210624 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2feb9d-5c02-47c4-992f-dd7c69549ffe admin admin] PATH_INFO: `/v3/projects/77d0588c9a2a4bc9b1fac934c2ca7f70/users/fe7c89dca78f4569ba1c144026ca0a95/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:43.224924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2feb9d-5c02-47c4-992f-dd7c69549ffe admin admin] RBAC: Authorizing `identity:create_grant(project_id=77d0588c9a2a4bc9b1fac934c2ca7f70, user_id=fe7c89dca78f4569ba1c144026ca0a95, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:43.227499 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2feb9d-5c02-47c4-992f-dd7c69549ffe admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:43.334031 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 949/1879] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:49:43 2022] PUT /identity/v3/projects/77d0588c9a2a4bc9b1fac934c2ca7f70/users/fe7c89dca78f4569ba1c144026ca0a95/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:49:43.348233 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0385f-0453-44bb-9596-97dd00db2fee None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:43.348587 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0385f-0453-44bb-9596-97dd00db2fee None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:43.348885 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0385f-0453-44bb-9596-97dd00db2fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:43.379421 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-7de0385f-0453-44bb-9596-97dd00db2fee None None] MFA Rules not processed for user `fe7c89dca78f4569ba1c144026ca0a95`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:49:43.380789 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-7de0385f-0453-44bb-9596-97dd00db2fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:43.447051 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 931/1880] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:49:43 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:49:43.451746 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2feb9d-5c02-47c4-992f-dd7c69549ffe admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:43.519976 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2feb9d-5c02-47c4-992f-dd7c69549ffe admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:43.525508 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4956d18d-f5ea-4801-bdfc-13aa13652bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:43.526116 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956d18d-f5ea-4801-bdfc-13aa13652bb9 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:43.526404 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956d18d-f5ea-4801-bdfc-13aa13652bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:43.526677 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956d18d-f5ea-4801-bdfc-13aa13652bb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:43.527325 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4956d18d-f5ea-4801-bdfc-13aa13652bb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:43.529297 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4956d18d-f5ea-4801-bdfc-13aa13652bb9 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:43.610224 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 950/1881] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:49:43 2022] POST /identity/v3/projects => generated 409 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:49:43.614537 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de0385f-0453-44bb-9596-97dd00db2fee None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:43.625642 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de0385f-0453-44bb-9596-97dd00db2fee None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:43.682121 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26488aa5-30d3-4fd3-a375-6775a60e1e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:43.682727 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26488aa5-30d3-4fd3-a375-6775a60e1e1c admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:43.683005 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26488aa5-30d3-4fd3-a375-6775a60e1e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:43.683275 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26488aa5-30d3-4fd3-a375-6775a60e1e1c admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:43.683909 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26488aa5-30d3-4fd3-a375-6775a60e1e1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:43.685860 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26488aa5-30d3-4fd3-a375-6775a60e1e1c admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:43.761711 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 932/1882] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:49:43 2022] POST /identity/v3/users => generated 311 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:49:43.765694 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4956d18d-f5ea-4801-bdfc-13aa13652bb9 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:43.784774 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4956d18d-f5ea-4801-bdfc-13aa13652bb9 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:43.789463 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf75782-7f55-43c2-a8a0-558b7e162afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:43.790356 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf75782-7f55-43c2-a8a0-558b7e162afb admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:43.790636 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf75782-7f55-43c2-a8a0-558b7e162afb admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:43.790906 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf75782-7f55-43c2-a8a0-558b7e162afb admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:43.791543 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf75782-7f55-43c2-a8a0-558b7e162afb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:43.793178 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf75782-7f55-43c2-a8a0-558b7e162afb admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:43.804444 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 951/1883] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:49:43.808730 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26488aa5-30d3-4fd3-a375-6775a60e1e1c admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:43.820116 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26488aa5-30d3-4fd3-a375-6775a60e1e1c admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:43.824775 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca82ca6-432d-4816-a785-a4cf7df07e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:43.825424 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca82ca6-432d-4816-a785-a4cf7df07e31 admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:43.825705 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca82ca6-432d-4816-a785-a4cf7df07e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:43.825979 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca82ca6-432d-4816-a785-a4cf7df07e31 admin admin] PATH_INFO: `/v3/projects/83a2279ef5db4931af661ed7ce43688b/users/18ed6518c55f4bc293f440ead11aae5f/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:43.841945 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca82ca6-432d-4816-a785-a4cf7df07e31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83a2279ef5db4931af661ed7ce43688b, user_id=18ed6518c55f4bc293f440ead11aae5f, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:43.844540 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca82ca6-432d-4816-a785-a4cf7df07e31 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:43.907027 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 933/1884] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:49:43 2022] PUT /identity/v3/projects/83a2279ef5db4931af661ed7ce43688b/users/18ed6518c55f4bc293f440ead11aae5f/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:49:43.911280 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf75782-7f55-43c2-a8a0-558b7e162afb admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:43.921393 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf75782-7f55-43c2-a8a0-558b7e162afb admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:43.942933 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:43.979055 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368974f2-1a15-4e8e-989c-4e5dbc254a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:43.979666 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368974f2-1a15-4e8e-989c-4e5dbc254a5e admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:43.979943 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368974f2-1a15-4e8e-989c-4e5dbc254a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:43.980220 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368974f2-1a15-4e8e-989c-4e5dbc254a5e admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:43.980888 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368974f2-1a15-4e8e-989c-4e5dbc254a5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:43.982508 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368974f2-1a15-4e8e-989c-4e5dbc254a5e admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:43.993198 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 952/1885] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:49:43.997638 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca82ca6-432d-4816-a785-a4cf7df07e31 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:44.003818 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:44.009279 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca82ca6-432d-4816-a785-a4cf7df07e31 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:44.013960 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbda1936-769c-4161-a4f1-2368061d0f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:44.017921 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda1936-769c-4161-a4f1-2368061d0f8d admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:44.017921 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda1936-769c-4161-a4f1-2368061d0f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:44.017921 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda1936-769c-4161-a4f1-2368061d0f8d admin admin] PATH_INFO: `/v3/projects/83a2279ef5db4931af661ed7ce43688b/users/18ed6518c55f4bc293f440ead11aae5f/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:44.017921 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbda1936-769c-4161-a4f1-2368061d0f8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=83a2279ef5db4931af661ed7ce43688b, user_id=18ed6518c55f4bc293f440ead11aae5f, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:44.019713 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbda1936-769c-4161-a4f1-2368061d0f8d admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:44.029933 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 934/1886] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:49:43 2022] PUT /identity/v3/projects/83a2279ef5db4931af661ed7ce43688b/users/18ed6518c55f4bc293f440ead11aae5f/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:49:44.040273 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1105dc-9cb9-4d1c-bf7e-e1b4571204bf None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:44.040505 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1105dc-9cb9-4d1c-bf7e-e1b4571204bf None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:44.040825 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1105dc-9cb9-4d1c-bf7e-e1b4571204bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:44.069720 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-8d1105dc-9cb9-4d1c-bf7e-e1b4571204bf None None] MFA Rules not processed for user `18ed6518c55f4bc293f440ead11aae5f`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:49:44.070976 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-8d1105dc-9cb9-4d1c-bf7e-e1b4571204bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:44.138956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 953/1887] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:49:44 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:49:44.150153 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbda1936-769c-4161-a4f1-2368061d0f8d admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:44.151803 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-fbda1936-769c-4161-a4f1-2368061d0f8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:44.218176 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbda1936-769c-4161-a4f1-2368061d0f8d admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:44.222624 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83a9690-917a-4768-bbfb-6973a6fbe4e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:44.223408 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a9690-917a-4768-bbfb-6973a6fbe4e4 service cinder] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:44.223897 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a9690-917a-4768-bbfb-6973a6fbe4e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:44.224346 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a9690-917a-4768-bbfb-6973a6fbe4e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:44.238886 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83a9690-917a-4768-bbfb-6973a6fbe4e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:44.240491 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83a9690-917a-4768-bbfb-6973a6fbe4e4 service cinder] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:44.253857 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 935/1888] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:49:44 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:49:44.706359 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1105dc-9cb9-4d1c-bf7e-e1b4571204bf None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:44.716341 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1105dc-9cb9-4d1c-bf7e-e1b4571204bf None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:44.720467 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea40445e-41aa-41c4-bab8-bc60fb851012 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:44.721166 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea40445e-41aa-41c4-bab8-bc60fb851012 service cinder] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:44.721500 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea40445e-41aa-41c4-bab8-bc60fb851012 service cinder] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:44.721770 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea40445e-41aa-41c4-bab8-bc60fb851012 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:44.792336 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea40445e-41aa-41c4-bab8-bc60fb851012 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:44.795105 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea40445e-41aa-41c4-bab8-bc60fb851012 service cinder] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:44.809740 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 954/1889] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:49:44 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:49:45.562085 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83a9690-917a-4768-bbfb-6973a6fbe4e4 service cinder] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:45.563571 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-c83a9690-917a-4768-bbfb-6973a6fbe4e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:45.649068 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83a9690-917a-4768-bbfb-6973a6fbe4e4 service cinder] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:45.653626 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9ad81-a5f1-4a20-93fd-c102aeb44fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:45.654565 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9ad81-a5f1-4a20-93fd-c102aeb44fe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:45.654908 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9ad81-a5f1-4a20-93fd-c102aeb44fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:45.655246 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9ad81-a5f1-4a20-93fd-c102aeb44fe8 admin admin] PATH_INFO: `/v3/users/b4518e453f394aac84faae55322bb969` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:45.656867 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad9ad81-a5f1-4a20-93fd-c102aeb44fe8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4518e453f394aac84faae55322bb969)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:45.659532 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad9ad81-a5f1-4a20-93fd-c102aeb44fe8 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:45.897227 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-8ad9ad81-a5f1-4a20-93fd-c102aeb44fe8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4518e453f394aac84faae55322bb969'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:45.911232 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-8ad9ad81-a5f1-4a20-93fd-c102aeb44fe8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4518e453f394aac84faae55322bb969'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:45.923351 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-8ad9ad81-a5f1-4a20-93fd-c102aeb44fe8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4518e453f394aac84faae55322bb969'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:46.034942 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-8ad9ad81-a5f1-4a20-93fd-c102aeb44fe8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4518e453f394aac84faae55322bb969'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:46.035594 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-8ad9ad81-a5f1-4a20-93fd-c102aeb44fe8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4518e453f394aac84faae55322bb969'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:46.054790 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 936/1890] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:49:45 2022] DELETE /identity/v3/users/b4518e453f394aac84faae55322bb969 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:49:46.059403 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acadd91-889f-4576-9e60-ab4a95663473 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:46.059871 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acadd91-889f-4576-9e60-ab4a95663473 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:46.079609 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acadd91-889f-4576-9e60-ab4a95663473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:46.080311 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-5acadd91-889f-4576-9e60-ab4a95663473 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:49:46.081589 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-5acadd91-889f-4576-9e60-ab4a95663473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:46.122551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:46.141926 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 955/1891] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:49:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:49:46.151220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9ad81-a5f1-4a20-93fd-c102aeb44fe8 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:46.152663 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-8ad9ad81-a5f1-4a20-93fd-c102aeb44fe8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:46.167012 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:46.210906 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9ad81-a5f1-4a20-93fd-c102aeb44fe8 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:46.214542 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29571fa-ef94-48e7-9b18-3fc0cadca754 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:46.215146 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29571fa-ef94-48e7-9b18-3fc0cadca754 service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:46.215427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29571fa-ef94-48e7-9b18-3fc0cadca754 service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:46.215705 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29571fa-ef94-48e7-9b18-3fc0cadca754 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:46.224659 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29571fa-ef94-48e7-9b18-3fc0cadca754 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:46.226128 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29571fa-ef94-48e7-9b18-3fc0cadca754 service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:46.235804 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 937/1892] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:49:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:49:46.827793 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acadd91-889f-4576-9e60-ab4a95663473 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:46.829534 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-5acadd91-889f-4576-9e60-ab4a95663473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:46.844467 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acadd91-889f-4576-9e60-ab4a95663473 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:46.851424 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f2902-454f-4dbb-bee0-6f82221b74fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:46.851957 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f2902-454f-4dbb-bee0-6f82221b74fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:46.852153 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f2902-454f-4dbb-bee0-6f82221b74fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:46.857514 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f2902-454f-4dbb-bee0-6f82221b74fb admin admin] PATH_INFO: `/v3/projects/d937c852390d400dbf119ef1d8e7abd5` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:46.857514 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5f2902-454f-4dbb-bee0-6f82221b74fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d937c852390d400dbf119ef1d8e7abd5)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:46.857514 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5f2902-454f-4dbb-bee0-6f82221b74fb admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:47.016280 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-7c5f2902-454f-4dbb-bee0-6f82221b74fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd937c852390d400dbf119ef1d8e7abd5', 'request_id': 'req-7c5f2902-454f-4dbb-bee0-6f82221b74fb'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:47.022927 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-7c5f2902-454f-4dbb-bee0-6f82221b74fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd937c852390d400dbf119ef1d8e7abd5', 'request_id': 'req-7c5f2902-454f-4dbb-bee0-6f82221b74fb'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:47.029382 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-7c5f2902-454f-4dbb-bee0-6f82221b74fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd937c852390d400dbf119ef1d8e7abd5', 'request_id': 'req-7c5f2902-454f-4dbb-bee0-6f82221b74fb'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:47.104030 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-7c5f2902-454f-4dbb-bee0-6f82221b74fb admin admin] The token cache is being invalidate because project d937c852390d400dbf119ef1d8e7abd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:49:47.104659 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-7c5f2902-454f-4dbb-bee0-6f82221b74fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:47.104995 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-7c5f2902-454f-4dbb-bee0-6f82221b74fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:47.108236 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 956/1893] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:49:46 2022] DELETE /identity/v3/projects/d937c852390d400dbf119ef1d8e7abd5 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:49:47.110668 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29571fa-ef94-48e7-9b18-3fc0cadca754 service neutron] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:47.121162 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29571fa-ef94-48e7-9b18-3fc0cadca754 service neutron] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:47.181094 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a0205e-8b3b-4215-8f24-0d64c52acd55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:47.181728 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a0205e-8b3b-4215-8f24-0d64c52acd55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:47.182006 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a0205e-8b3b-4215-8f24-0d64c52acd55 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:47.182279 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a0205e-8b3b-4215-8f24-0d64c52acd55 admin admin] PATH_INFO: `/v3/users/92feef7187da4ffe8f292de2036bc885` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:47.183390 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a0205e-8b3b-4215-8f24-0d64c52acd55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92feef7187da4ffe8f292de2036bc885)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:47.185772 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a0205e-8b3b-4215-8f24-0d64c52acd55 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:47.415688 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-83a0205e-8b3b-4215-8f24-0d64c52acd55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92feef7187da4ffe8f292de2036bc885'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:47.428784 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-83a0205e-8b3b-4215-8f24-0d64c52acd55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92feef7187da4ffe8f292de2036bc885'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:47.436010 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-83a0205e-8b3b-4215-8f24-0d64c52acd55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92feef7187da4ffe8f292de2036bc885'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:47.558548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-83a0205e-8b3b-4215-8f24-0d64c52acd55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92feef7187da4ffe8f292de2036bc885'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:47.559270 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-83a0205e-8b3b-4215-8f24-0d64c52acd55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92feef7187da4ffe8f292de2036bc885'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:47.569159 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 938/1894] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:49:47 2022] DELETE /identity/v3/users/92feef7187da4ffe8f292de2036bc885 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:49:48.904111 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f2902-454f-4dbb-bee0-6f82221b74fb admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:48.905758 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-7c5f2902-454f-4dbb-bee0-6f82221b74fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:48.929561 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f2902-454f-4dbb-bee0-6f82221b74fb admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:49.005960 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d73c5d-3d39-49c3-985a-bb706d7e02b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:49.006530 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d73c5d-3d39-49c3-985a-bb706d7e02b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:49.006807 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d73c5d-3d39-49c3-985a-bb706d7e02b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:49.007081 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d73c5d-3d39-49c3-985a-bb706d7e02b8 admin admin] PATH_INFO: `/v3/projects/170d841910bf4f0aa0d1f6ac00ebf488` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:49.008448 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d73c5d-3d39-49c3-985a-bb706d7e02b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=170d841910bf4f0aa0d1f6ac00ebf488)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:49.010379 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d73c5d-3d39-49c3-985a-bb706d7e02b8 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:49.130176 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d7d73c5d-3d39-49c3-985a-bb706d7e02b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '170d841910bf4f0aa0d1f6ac00ebf488', 'request_id': 'req-d7d73c5d-3d39-49c3-985a-bb706d7e02b8'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:49.138838 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d7d73c5d-3d39-49c3-985a-bb706d7e02b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '170d841910bf4f0aa0d1f6ac00ebf488', 'request_id': 'req-d7d73c5d-3d39-49c3-985a-bb706d7e02b8'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:49.146518 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d7d73c5d-3d39-49c3-985a-bb706d7e02b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '170d841910bf4f0aa0d1f6ac00ebf488', 'request_id': 'req-d7d73c5d-3d39-49c3-985a-bb706d7e02b8'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:49.227627 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:49.234214 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d7d73c5d-3d39-49c3-985a-bb706d7e02b8 admin admin] The token cache is being invalidate because project 170d841910bf4f0aa0d1f6ac00ebf488 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:49:49.234214 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d7d73c5d-3d39-49c3-985a-bb706d7e02b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:49.234763 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d7d73c5d-3d39-49c3-985a-bb706d7e02b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:49.236468 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 957/1895] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:49:48 2022] DELETE /identity/v3/projects/170d841910bf4f0aa0d1f6ac00ebf488 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:49:49.242423 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de442b3-609d-4580-886e-2e7ecb490816 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:49.242915 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de442b3-609d-4580-886e-2e7ecb490816 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:49.242993 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de442b3-609d-4580-886e-2e7ecb490816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:49.265673 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-0de442b3-609d-4580-886e-2e7ecb490816 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:49:49.267098 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-0de442b3-609d-4580-886e-2e7ecb490816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:49.302808 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:49.335469 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 939/1896] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:49:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:49:49.343359 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c22e3-925c-420c-bdc5-e3a22d762a91 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:49.347598 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c22e3-925c-420c-bdc5-e3a22d762a91 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:49.347953 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c22e3-925c-420c-bdc5-e3a22d762a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:49.377049 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-364c22e3-925c-420c-bdc5-e3a22d762a91 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:49:49.377049 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-364c22e3-925c-420c-bdc5-e3a22d762a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:49.384301 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 958/1897] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:49:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:49:49.388695 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de442b3-609d-4580-886e-2e7ecb490816 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:49.399418 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de442b3-609d-4580-886e-2e7ecb490816 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:49.405039 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5d783-e3b2-4e56-b897-90f4ede769ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:49.405689 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5d783-e3b2-4e56-b897-90f4ede769ab admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:49.405978 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5d783-e3b2-4e56-b897-90f4ede769ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:49.406252 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5d783-e3b2-4e56-b897-90f4ede769ab admin admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:49.406915 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe5d783-e3b2-4e56-b897-90f4ede769ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:49:49.407325 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe5d783-e3b2-4e56-b897-90f4ede769ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:49.409933 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe5d783-e3b2-4e56-b897-90f4ede769ab admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:49.424236 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 940/1898] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:49:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:49:49.428591 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c22e3-925c-420c-bdc5-e3a22d762a91 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:49.438473 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c22e3-925c-420c-bdc5-e3a22d762a91 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:49.444884 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f22995d-75e0-4536-92e5-dd457d9f750d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:49.448881 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f22995d-75e0-4536-92e5-dd457d9f750d admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:49.448881 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f22995d-75e0-4536-92e5-dd457d9f750d admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:49.448881 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f22995d-75e0-4536-92e5-dd457d9f750d admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:49.448881 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f22995d-75e0-4536-92e5-dd457d9f750d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:49.448881 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f22995d-75e0-4536-92e5-dd457d9f750d admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:49.532531 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 959/1899] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:49:49 2022] POST /identity/v3/projects => generated 387 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:49:49.538291 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5d783-e3b2-4e56-b897-90f4ede769ab admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:49.556332 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5d783-e3b2-4e56-b897-90f4ede769ab admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:49.622103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbb5bc7-d24e-4c27-9732-7553d207a054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:49.622879 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbb5bc7-d24e-4c27-9732-7553d207a054 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:49.623249 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbb5bc7-d24e-4c27-9732-7553d207a054 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:49.623604 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbb5bc7-d24e-4c27-9732-7553d207a054 admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:49.624412 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbb5bc7-d24e-4c27-9732-7553d207a054 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:49.626662 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbb5bc7-d24e-4c27-9732-7553d207a054 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:49.760745 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 941/1900] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:49:49 2022] POST /identity/v3/users => generated 294 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:49:49.770801 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f22995d-75e0-4536-92e5-dd457d9f750d admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:49.773189 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f22995d-75e0-4536-92e5-dd457d9f750d admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:49.778733 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bd76c6-0c27-4bed-918a-30d240dd5581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:49.779420 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd76c6-0c27-4bed-918a-30d240dd5581 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:49.779888 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd76c6-0c27-4bed-918a-30d240dd5581 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:49.780227 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd76c6-0c27-4bed-918a-30d240dd5581 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:49.780985 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bd76c6-0c27-4bed-918a-30d240dd5581 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:49.782740 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bd76c6-0c27-4bed-918a-30d240dd5581 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:49.793730 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 960/1901] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:49:49.797807 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbb5bc7-d24e-4c27-9732-7553d207a054 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:49.809389 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbb5bc7-d24e-4c27-9732-7553d207a054 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:49.814993 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f601b8f2-3f8f-48e5-af8b-46f006ddcc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:49.817070 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601b8f2-3f8f-48e5-af8b-46f006ddcc92 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:49.817070 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601b8f2-3f8f-48e5-af8b-46f006ddcc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:49.817279 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601b8f2-3f8f-48e5-af8b-46f006ddcc92 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:49.817837 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f601b8f2-3f8f-48e5-af8b-46f006ddcc92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:49.819505 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f601b8f2-3f8f-48e5-af8b-46f006ddcc92 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:49.835343 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 942/1902] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:49:49.836465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bd76c6-0c27-4bed-918a-30d240dd5581 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:49.848024 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bd76c6-0c27-4bed-918a-30d240dd5581 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:49.852804 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb228ed-fc5d-4c4b-b6de-94c838805137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:49.853470 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb228ed-fc5d-4c4b-b6de-94c838805137 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:49.853767 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb228ed-fc5d-4c4b-b6de-94c838805137 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:49.854046 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb228ed-fc5d-4c4b-b6de-94c838805137 admin admin] PATH_INFO: `/v3/projects/e3ad4fc405c5475492a194d79dedc957/users/53a5334d87b2402196b012ec0059e088/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:49.875320 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb228ed-fc5d-4c4b-b6de-94c838805137 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3ad4fc405c5475492a194d79dedc957, user_id=53a5334d87b2402196b012ec0059e088, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:49.878963 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb228ed-fc5d-4c4b-b6de-94c838805137 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:49.950206 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 961/1903] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:49:49 2022] PUT /identity/v3/projects/e3ad4fc405c5475492a194d79dedc957/users/53a5334d87b2402196b012ec0059e088/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:49:49.962672 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49464db6-75e3-431a-adf0-edbf394a4973 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:49.962983 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49464db6-75e3-431a-adf0-edbf394a4973 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:49.963255 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49464db6-75e3-431a-adf0-edbf394a4973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:49.993596 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-49464db6-75e3-431a-adf0-edbf394a4973 None None] MFA Rules not processed for user `53a5334d87b2402196b012ec0059e088`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:49:49.994899 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-49464db6-75e3-431a-adf0-edbf394a4973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:50.062980 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 943/1904] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:49:49 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:49:50.068108 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb228ed-fc5d-4c4b-b6de-94c838805137 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:50.134863 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb228ed-fc5d-4c4b-b6de-94c838805137 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:50.139642 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490cf243-3881-4e90-ac4c-49bcfe5d3da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:50.140381 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490cf243-3881-4e90-ac4c-49bcfe5d3da9 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:50.140499 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490cf243-3881-4e90-ac4c-49bcfe5d3da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:50.140804 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490cf243-3881-4e90-ac4c-49bcfe5d3da9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:50.141433 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490cf243-3881-4e90-ac4c-49bcfe5d3da9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:50.143415 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490cf243-3881-4e90-ac4c-49bcfe5d3da9 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:50.250879 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 962/1905] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:49:50 2022] POST /identity/v3/projects => generated 385 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:49:50.255202 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49464db6-75e3-431a-adf0-edbf394a4973 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:50.268301 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49464db6-75e3-431a-adf0-edbf394a4973 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:50.303649 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:50.338345 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60553fe-176d-4b04-94cd-54c99156b14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:50.339138 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60553fe-176d-4b04-94cd-54c99156b14c admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:50.339427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60553fe-176d-4b04-94cd-54c99156b14c admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:50.339703 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60553fe-176d-4b04-94cd-54c99156b14c admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:50.340344 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60553fe-176d-4b04-94cd-54c99156b14c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:50.342366 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60553fe-176d-4b04-94cd-54c99156b14c admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:50.436329 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 944/1906] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:49:50 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:49:50.436657 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490cf243-3881-4e90-ac4c-49bcfe5d3da9 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:50.447686 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490cf243-3881-4e90-ac4c-49bcfe5d3da9 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:50.452673 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64731a5d-9fbc-4621-b0c1-96d78639da6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:50.453945 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64731a5d-9fbc-4621-b0c1-96d78639da6b admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:50.454539 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64731a5d-9fbc-4621-b0c1-96d78639da6b admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:50.454800 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64731a5d-9fbc-4621-b0c1-96d78639da6b admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:50.455457 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64731a5d-9fbc-4621-b0c1-96d78639da6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:50.457136 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64731a5d-9fbc-4621-b0c1-96d78639da6b admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:50.468500 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 963/1907] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:49:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:49:50.473072 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60553fe-176d-4b04-94cd-54c99156b14c admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:50.482883 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60553fe-176d-4b04-94cd-54c99156b14c admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:50.488212 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26205e27-872b-4439-b5e8-c889a7be6b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:50.488777 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26205e27-872b-4439-b5e8-c889a7be6b6b admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:50.493769 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26205e27-872b-4439-b5e8-c889a7be6b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:50.493769 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26205e27-872b-4439-b5e8-c889a7be6b6b admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:50.493769 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26205e27-872b-4439-b5e8-c889a7be6b6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:50.493769 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26205e27-872b-4439-b5e8-c889a7be6b6b admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:50.501287 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 945/1908] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:49:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:49:50.505701 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64731a5d-9fbc-4621-b0c1-96d78639da6b admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:50.522780 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64731a5d-9fbc-4621-b0c1-96d78639da6b admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:50.522959 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9020e0-0c96-4276-b539-c244adea0929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:50.523668 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9020e0-0c96-4276-b539-c244adea0929 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:50.523952 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9020e0-0c96-4276-b539-c244adea0929 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:50.524328 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9020e0-0c96-4276-b539-c244adea0929 admin admin] PATH_INFO: `/v3/projects/ed3f681f20d1495b939a9a7af243fcce/users/321c5f768791446fbb20bf0d8340d9d9/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:50.549056 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9020e0-0c96-4276-b539-c244adea0929 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed3f681f20d1495b939a9a7af243fcce, user_id=321c5f768791446fbb20bf0d8340d9d9, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:50.553036 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9020e0-0c96-4276-b539-c244adea0929 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:50.655289 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 964/1909] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:49:50 2022] PUT /identity/v3/projects/ed3f681f20d1495b939a9a7af243fcce/users/321c5f768791446fbb20bf0d8340d9d9/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:49:50.667576 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0cce1a-1f4f-403e-b1a4-1ee82ec31385 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:50.667576 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0cce1a-1f4f-403e-b1a4-1ee82ec31385 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:50.667576 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0cce1a-1f4f-403e-b1a4-1ee82ec31385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:50.718324 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-bb0cce1a-1f4f-403e-b1a4-1ee82ec31385 None None] MFA Rules not processed for user `321c5f768791446fbb20bf0d8340d9d9`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:49:50.719589 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-bb0cce1a-1f4f-403e-b1a4-1ee82ec31385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:50.815731 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9020e0-0c96-4276-b539-c244adea0929 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:50.816327 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 946/1910] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:49:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:49:50.893664 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:50.910605 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9020e0-0c96-4276-b539-c244adea0929 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:50.915870 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c8b90a-0137-4d7c-a45c-f3a7716371f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:50.916681 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8b90a-0137-4d7c-a45c-f3a7716371f9 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:50.916998 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8b90a-0137-4d7c-a45c-f3a7716371f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:50.917274 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8b90a-0137-4d7c-a45c-f3a7716371f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:50.917917 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c8b90a-0137-4d7c-a45c-f3a7716371f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:50.920158 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c8b90a-0137-4d7c-a45c-f3a7716371f9 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:51.034599 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 965/1911] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:49:50 2022] POST /identity/v3/projects => generated 387 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:49:51.038694 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0cce1a-1f4f-403e-b1a4-1ee82ec31385 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:51.050077 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0cce1a-1f4f-403e-b1a4-1ee82ec31385 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:51.110874 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce0c44-c32d-4c1b-9130-18d9fcfb99a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:51.111480 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce0c44-c32d-4c1b-9130-18d9fcfb99a6 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:51.111758 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce0c44-c32d-4c1b-9130-18d9fcfb99a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:51.112766 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce0c44-c32d-4c1b-9130-18d9fcfb99a6 admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:51.113400 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ce0c44-c32d-4c1b-9130-18d9fcfb99a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:51.115306 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ce0c44-c32d-4c1b-9130-18d9fcfb99a6 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:51.272516 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 947/1912] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:49:51 2022] POST /identity/v3/users => generated 300 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:49:51.273762 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c8b90a-0137-4d7c-a45c-f3a7716371f9 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:51.285457 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c8b90a-0137-4d7c-a45c-f3a7716371f9 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:51.291464 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4311a6a-6632-4c8e-87f5-883e53ba98ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:51.292574 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4311a6a-6632-4c8e-87f5-883e53ba98ba admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:51.293181 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4311a6a-6632-4c8e-87f5-883e53ba98ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:51.298271 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4311a6a-6632-4c8e-87f5-883e53ba98ba admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:51.298271 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4311a6a-6632-4c8e-87f5-883e53ba98ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:51.298271 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4311a6a-6632-4c8e-87f5-883e53ba98ba admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:51.309777 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 966/1913] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:49:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:49:51.314001 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce0c44-c32d-4c1b-9130-18d9fcfb99a6 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:51.324588 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce0c44-c32d-4c1b-9130-18d9fcfb99a6 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:51.330203 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b53991-4754-4fcd-8225-a5accfd4bf7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:51.331670 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b53991-4754-4fcd-8225-a5accfd4bf7a admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:51.332506 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b53991-4754-4fcd-8225-a5accfd4bf7a admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:51.333041 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b53991-4754-4fcd-8225-a5accfd4bf7a admin admin] PATH_INFO: `/v3/projects/5b433a2329974d1aa6c524201ce21af4/users/f1afc5154fe6487580a8449a37b92f02/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:51.354093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b53991-4754-4fcd-8225-a5accfd4bf7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b433a2329974d1aa6c524201ce21af4, user_id=f1afc5154fe6487580a8449a37b92f02, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:51.358122 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b53991-4754-4fcd-8225-a5accfd4bf7a admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:51.466641 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 948/1914] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:49:51 2022] PUT /identity/v3/projects/5b433a2329974d1aa6c524201ce21af4/users/f1afc5154fe6487580a8449a37b92f02/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:49:51.473194 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4311a6a-6632-4c8e-87f5-883e53ba98ba admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:51.485287 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4311a6a-6632-4c8e-87f5-883e53ba98ba admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:51.583380 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d2b890-fe56-4fac-a325-7238c9754f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:51.584333 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d2b890-fe56-4fac-a325-7238c9754f20 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:51.587674 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d2b890-fe56-4fac-a325-7238c9754f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:51.588618 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d2b890-fe56-4fac-a325-7238c9754f20 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:51.590103 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d2b890-fe56-4fac-a325-7238c9754f20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:51.592611 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d2b890-fe56-4fac-a325-7238c9754f20 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:51.610702 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 967/1915] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:49:51 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:49:51.611422 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b53991-4754-4fcd-8225-a5accfd4bf7a admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:51.626585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:51.635405 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b53991-4754-4fcd-8225-a5accfd4bf7a admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:51.640906 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b1b5d7-c66b-4338-a746-dc470452e3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:51.641649 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b1b5d7-c66b-4338-a746-dc470452e3b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:51.641947 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b1b5d7-c66b-4338-a746-dc470452e3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:51.642258 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b1b5d7-c66b-4338-a746-dc470452e3b6 admin admin] PATH_INFO: `/v3/projects/5b433a2329974d1aa6c524201ce21af4/users/f1afc5154fe6487580a8449a37b92f02/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:51.646076 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b1b5d7-c66b-4338-a746-dc470452e3b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b433a2329974d1aa6c524201ce21af4, user_id=f1afc5154fe6487580a8449a37b92f02, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:51.649832 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b1b5d7-c66b-4338-a746-dc470452e3b6 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:51.668280 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 949/1916] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:49:51 2022] PUT /identity/v3/projects/5b433a2329974d1aa6c524201ce21af4/users/f1afc5154fe6487580a8449a37b92f02/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:49:51.678276 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9123d-6147-4811-b2c2-b973a04c831f None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:51.678276 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9123d-6147-4811-b2c2-b973a04c831f None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:51.678602 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9123d-6147-4811-b2c2-b973a04c831f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:51.718650 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-1fd9123d-6147-4811-b2c2-b973a04c831f None None] MFA Rules not processed for user `f1afc5154fe6487580a8449a37b92f02`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:49:51.720236 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-1fd9123d-6147-4811-b2c2-b973a04c831f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:51.846623 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 968/1917] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:49:51 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:49:52.012426 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b1b5d7-c66b-4338-a746-dc470452e3b6 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:52.014803 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-a4b1b5d7-c66b-4338-a746-dc470452e3b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:52.083208 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b1b5d7-c66b-4338-a746-dc470452e3b6 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:52.087911 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0059f2-c6c5-4730-8ec7-822614af0d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:52.088544 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0059f2-c6c5-4730-8ec7-822614af0d26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:52.088862 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0059f2-c6c5-4730-8ec7-822614af0d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:52.089138 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0059f2-c6c5-4730-8ec7-822614af0d26 admin admin] PATH_INFO: `/v3/users/d43636bf86e441b5b72d00f85e01eac7` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:52.090258 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0059f2-c6c5-4730-8ec7-822614af0d26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d43636bf86e441b5b72d00f85e01eac7)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:52.106375 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0059f2-c6c5-4730-8ec7-822614af0d26 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:52.198216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd9123d-6147-4811-b2c2-b973a04c831f None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:52.199978 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-1fd9123d-6147-4811-b2c2-b973a04c831f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:52.246003 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:52.264013 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd9123d-6147-4811-b2c2-b973a04c831f None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:52.268204 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2547c70e-9c3d-4afa-b345-404d1e60bf2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:52.269011 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547c70e-9c3d-4afa-b345-404d1e60bf2a service cinder] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:52.269286 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547c70e-9c3d-4afa-b345-404d1e60bf2a service cinder] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:52.269562 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547c70e-9c3d-4afa-b345-404d1e60bf2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:52.278985 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2547c70e-9c3d-4afa-b345-404d1e60bf2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:52.280390 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2547c70e-9c3d-4afa-b345-404d1e60bf2a service cinder] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:52.290226 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 969/1918] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:49:52 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:49:52.354778 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-fc0059f2-c6c5-4730-8ec7-822614af0d26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd43636bf86e441b5b72d00f85e01eac7'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:52.367409 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-fc0059f2-c6c5-4730-8ec7-822614af0d26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd43636bf86e441b5b72d00f85e01eac7'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:52.374580 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-fc0059f2-c6c5-4730-8ec7-822614af0d26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd43636bf86e441b5b72d00f85e01eac7'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:52.460226 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-fc0059f2-c6c5-4730-8ec7-822614af0d26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd43636bf86e441b5b72d00f85e01eac7'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:52.461220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-fc0059f2-c6c5-4730-8ec7-822614af0d26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd43636bf86e441b5b72d00f85e01eac7'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:52.473741 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 950/1919] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:49:52 2022] DELETE /identity/v3/users/d43636bf86e441b5b72d00f85e01eac7 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:49:52.817764 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2547c70e-9c3d-4afa-b345-404d1e60bf2a service cinder] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:52.830065 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2547c70e-9c3d-4afa-b345-404d1e60bf2a service cinder] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:52.926434 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d72ae-8479-46bf-97e6-b2fc415b891a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:52.931328 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d72ae-8479-46bf-97e6-b2fc415b891a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:52.931328 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d72ae-8479-46bf-97e6-b2fc415b891a admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:52.931328 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d72ae-8479-46bf-97e6-b2fc415b891a admin admin] PATH_INFO: `/v3/projects/cedceef6152643f7875eff039186cd55` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:52.931328 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d72ae-8479-46bf-97e6-b2fc415b891a admin admin] RBAC: Authorizing `identity:delete_project(project_id=cedceef6152643f7875eff039186cd55)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:52.932422 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d72ae-8479-46bf-97e6-b2fc415b891a admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:53.116397 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-407d72ae-8479-46bf-97e6-b2fc415b891a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cedceef6152643f7875eff039186cd55', 'request_id': 'req-407d72ae-8479-46bf-97e6-b2fc415b891a'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:53.122461 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-407d72ae-8479-46bf-97e6-b2fc415b891a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cedceef6152643f7875eff039186cd55', 'request_id': 'req-407d72ae-8479-46bf-97e6-b2fc415b891a'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:53.128937 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-407d72ae-8479-46bf-97e6-b2fc415b891a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cedceef6152643f7875eff039186cd55', 'request_id': 'req-407d72ae-8479-46bf-97e6-b2fc415b891a'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:53.210057 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-407d72ae-8479-46bf-97e6-b2fc415b891a admin admin] The token cache is being invalidate because project cedceef6152643f7875eff039186cd55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:49:53.210717 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-407d72ae-8479-46bf-97e6-b2fc415b891a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:53.211057 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-407d72ae-8479-46bf-97e6-b2fc415b891a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:53.212585 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 970/1920] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:49:52 2022] DELETE /identity/v3/projects/cedceef6152643f7875eff039186cd55 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:49:53.217822 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed597f5-27c9-44f9-809c-f9f6daebdd7f None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:53.218111 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed597f5-27c9-44f9-809c-f9f6daebdd7f None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:53.218388 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed597f5-27c9-44f9-809c-f9f6daebdd7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:53.239257 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-8ed597f5-27c9-44f9-809c-f9f6daebdd7f None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:49:53.240502 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-8ed597f5-27c9-44f9-809c-f9f6daebdd7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:53.303797 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 951/1921] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:49:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:49:53.312078 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb6a54-76a9-40c1-9fe6-062b068a31e6 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:53.312381 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb6a54-76a9-40c1-9fe6-062b068a31e6 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:53.312735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb6a54-76a9-40c1-9fe6-062b068a31e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:53.334367 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-effb6a54-76a9-40c1-9fe6-062b068a31e6 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:49:53.335638 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-effb6a54-76a9-40c1-9fe6-062b068a31e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:53.343699 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 971/1922] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:49:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:49:53.348238 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed597f5-27c9-44f9-809c-f9f6daebdd7f None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:53.358645 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed597f5-27c9-44f9-809c-f9f6daebdd7f None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:53.364288 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec7e412-5300-46b7-a12d-272deb108b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:53.364951 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec7e412-5300-46b7-a12d-272deb108b10 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:53.365232 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec7e412-5300-46b7-a12d-272deb108b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:53.365504 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec7e412-5300-46b7-a12d-272deb108b10 admin admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:53.366157 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec7e412-5300-46b7-a12d-272deb108b10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:49:53.366480 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec7e412-5300-46b7-a12d-272deb108b10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:53.368133 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec7e412-5300-46b7-a12d-272deb108b10 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:53.370834 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:53.381155 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 952/1923] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:49:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:49:53.385620 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb6a54-76a9-40c1-9fe6-062b068a31e6 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:53.395541 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb6a54-76a9-40c1-9fe6-062b068a31e6 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:53.400395 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d5b14b-ac52-48af-8dde-4d34f2ab7556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:53.401053 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d5b14b-ac52-48af-8dde-4d34f2ab7556 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:53.401335 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d5b14b-ac52-48af-8dde-4d34f2ab7556 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:53.401608 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d5b14b-ac52-48af-8dde-4d34f2ab7556 admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:53.402233 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d5b14b-ac52-48af-8dde-4d34f2ab7556 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:53.404197 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d5b14b-ac52-48af-8dde-4d34f2ab7556 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:53.473216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 972/1924] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:49:53 2022] POST /identity/v3/projects => generated 367 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:49:53.477430 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec7e412-5300-46b7-a12d-272deb108b10 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:53.488039 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec7e412-5300-46b7-a12d-272deb108b10 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:53.549008 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae45a99-b9d3-4786-b611-39df0059ca02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:53.550245 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae45a99-b9d3-4786-b611-39df0059ca02 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:53.550527 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae45a99-b9d3-4786-b611-39df0059ca02 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:53.550807 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae45a99-b9d3-4786-b611-39df0059ca02 admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:53.551449 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae45a99-b9d3-4786-b611-39df0059ca02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:53.553444 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae45a99-b9d3-4786-b611-39df0059ca02 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:53.638246 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 953/1925] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:49:53 2022] POST /identity/v3/users => generated 284 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:49:53.642594 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d5b14b-ac52-48af-8dde-4d34f2ab7556 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:53.654863 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d5b14b-ac52-48af-8dde-4d34f2ab7556 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:53.659958 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718c292-413a-4dc8-9e86-54c550cf4e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:53.660757 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718c292-413a-4dc8-9e86-54c550cf4e2c admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:53.660875 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718c292-413a-4dc8-9e86-54c550cf4e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:53.661145 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718c292-413a-4dc8-9e86-54c550cf4e2c admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:53.661800 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1718c292-413a-4dc8-9e86-54c550cf4e2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:53.663399 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1718c292-413a-4dc8-9e86-54c550cf4e2c admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:53.674029 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 973/1926] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:49:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:49:53.678659 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae45a99-b9d3-4786-b611-39df0059ca02 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:53.691152 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae45a99-b9d3-4786-b611-39df0059ca02 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:53.695380 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d32f001-2d18-4b76-a3e1-9ab86b9c58f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:53.695981 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d32f001-2d18-4b76-a3e1-9ab86b9c58f2 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:53.696386 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d32f001-2d18-4b76-a3e1-9ab86b9c58f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:53.696661 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d32f001-2d18-4b76-a3e1-9ab86b9c58f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:53.697337 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d32f001-2d18-4b76-a3e1-9ab86b9c58f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:53.698973 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d32f001-2d18-4b76-a3e1-9ab86b9c58f2 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:53.711268 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 954/1927] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:49:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:49:53.715669 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718c292-413a-4dc8-9e86-54c550cf4e2c admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:53.726539 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718c292-413a-4dc8-9e86-54c550cf4e2c admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:53.731230 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebabfa23-4497-41b6-bf27-aca4a36c8a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:53.731959 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebabfa23-4497-41b6-bf27-aca4a36c8a42 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:53.732157 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebabfa23-4497-41b6-bf27-aca4a36c8a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:53.732435 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebabfa23-4497-41b6-bf27-aca4a36c8a42 admin admin] PATH_INFO: `/v3/projects/109d626ecc0348769c7c3e8bfd0322d8/users/26c148d4a1fc40969d0a29cb8aea1a56/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:53.749494 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebabfa23-4497-41b6-bf27-aca4a36c8a42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=109d626ecc0348769c7c3e8bfd0322d8, user_id=26c148d4a1fc40969d0a29cb8aea1a56, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:53.752351 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebabfa23-4497-41b6-bf27-aca4a36c8a42 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:53.762723 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d32f001-2d18-4b76-a3e1-9ab86b9c58f2 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:53.764625 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-0d32f001-2d18-4b76-a3e1-9ab86b9c58f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:53.827354 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d32f001-2d18-4b76-a3e1-9ab86b9c58f2 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:53.831677 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbba029-42af-487b-aa40-38b1a427dfc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:53.832289 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbba029-42af-487b-aa40-38b1a427dfc8 service cinder] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:53.832568 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbba029-42af-487b-aa40-38b1a427dfc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:53.832886 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbba029-42af-487b-aa40-38b1a427dfc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:53.834630 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-2fbba029-42af-487b-aa40-38b1a427dfc8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:53.879091 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 974/1928] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:49:53 2022] PUT /identity/v3/projects/109d626ecc0348769c7c3e8bfd0322d8/users/26c148d4a1fc40969d0a29cb8aea1a56/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:49:53.890362 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ee50c-19d8-49b0-a07f-e82467aa21dc None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:53.890647 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ee50c-19d8-49b0-a07f-e82467aa21dc None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:53.890921 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ee50c-19d8-49b0-a07f-e82467aa21dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:53.895161 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbba029-42af-487b-aa40-38b1a427dfc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:53.896605 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbba029-42af-487b-aa40-38b1a427dfc8 service cinder] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:53.906337 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 955/1929] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:49:53 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:49:53.912140 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:53.928231 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-815ee50c-19d8-49b0-a07f-e82467aa21dc None None] MFA Rules not processed for user `26c148d4a1fc40969d0a29cb8aea1a56`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:49:53.929541 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-815ee50c-19d8-49b0-a07f-e82467aa21dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:54.014634 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 975/1930] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:49:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:49:54.019345 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbba029-42af-487b-aa40-38b1a427dfc8 service cinder] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:54.034438 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbba029-42af-487b-aa40-38b1a427dfc8 service cinder] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:54.099694 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:54.146722 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098876e0-729b-4973-8db9-4107184d6f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:54.147345 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098876e0-729b-4973-8db9-4107184d6f94 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:54.147625 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098876e0-729b-4973-8db9-4107184d6f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:54.148003 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098876e0-729b-4973-8db9-4107184d6f94 admin admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:54.148654 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098876e0-729b-4973-8db9-4107184d6f94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:54.151089 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098876e0-729b-4973-8db9-4107184d6f94 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:54.211120 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ee50c-19d8-49b0-a07f-e82467aa21dc None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:54.214507 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-815ee50c-19d8-49b0-a07f-e82467aa21dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:54.233121 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ee50c-19d8-49b0-a07f-e82467aa21dc None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:54.244587 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb5174-c2c3-483e-8a05-767b384ebf91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:54.245303 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb5174-c2c3-483e-8a05-767b384ebf91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:54.245583 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb5174-c2c3-483e-8a05-767b384ebf91 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:54.245896 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb5174-c2c3-483e-8a05-767b384ebf91 admin admin] PATH_INFO: `/v3/users/d4d4dc994f9744ab990866ae8def199d` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:54.247141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fb5174-c2c3-483e-8a05-767b384ebf91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4d4dc994f9744ab990866ae8def199d)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:54.250314 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fb5174-c2c3-483e-8a05-767b384ebf91 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:54.290714 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 956/1931] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:49:54 2022] POST /identity/v3/projects => generated 367 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:49:54.296287 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098876e0-729b-4973-8db9-4107184d6f94 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:54.383474 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098876e0-729b-4973-8db9-4107184d6f94 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:54.388327 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1227f929-3d38-4cb3-b35c-c2325dad5262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:54.388980 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227f929-3d38-4cb3-b35c-c2325dad5262 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:54.390774 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227f929-3d38-4cb3-b35c-c2325dad5262 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:54.391057 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227f929-3d38-4cb3-b35c-c2325dad5262 admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:54.394230 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1227f929-3d38-4cb3-b35c-c2325dad5262 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:54.397697 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1227f929-3d38-4cb3-b35c-c2325dad5262 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:54.470619 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-43fb5174-c2c3-483e-8a05-767b384ebf91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4d4dc994f9744ab990866ae8def199d'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:54.512884 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 957/1932] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:49:54 2022] POST /identity/v3/users => generated 290 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:49:54.520944 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1227f929-3d38-4cb3-b35c-c2325dad5262 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:54.536306 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1227f929-3d38-4cb3-b35c-c2325dad5262 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:54.544317 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d6e62-1b56-4e88-a375-2e7246cafb4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:54.545674 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d6e62-1b56-4e88-a375-2e7246cafb4b admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:54.546025 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d6e62-1b56-4e88-a375-2e7246cafb4b admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:54.546307 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d6e62-1b56-4e88-a375-2e7246cafb4b admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:54.547000 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09d6e62-1b56-4e88-a375-2e7246cafb4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:54.551051 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09d6e62-1b56-4e88-a375-2e7246cafb4b admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:54.564817 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 958/1933] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:49:54 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:49:54.569410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d6e62-1b56-4e88-a375-2e7246cafb4b admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:54.589423 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d6e62-1b56-4e88-a375-2e7246cafb4b admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:54.594081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2c6947-114f-4401-9789-9bfb9b40dc37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:54.594734 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c6947-114f-4401-9789-9bfb9b40dc37 admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:54.595012 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c6947-114f-4401-9789-9bfb9b40dc37 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:54.595279 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c6947-114f-4401-9789-9bfb9b40dc37 admin admin] PATH_INFO: `/v3/projects/c9acb65d67d74d5aa835a0840faf5336/users/167bf96ff5894771ab52d22ddb3c5b07/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:54.598491 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-43fb5174-c2c3-483e-8a05-767b384ebf91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4d4dc994f9744ab990866ae8def199d'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:54.616216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-43fb5174-c2c3-483e-8a05-767b384ebf91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4d4dc994f9744ab990866ae8def199d'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:54.616452 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2c6947-114f-4401-9789-9bfb9b40dc37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9acb65d67d74d5aa835a0840faf5336, user_id=167bf96ff5894771ab52d22ddb3c5b07, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:54.621956 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2c6947-114f-4401-9789-9bfb9b40dc37 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:54.623802 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-43fb5174-c2c3-483e-8a05-767b384ebf91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4d4dc994f9744ab990866ae8def199d'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:54.626251 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-43fb5174-c2c3-483e-8a05-767b384ebf91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4d4dc994f9744ab990866ae8def199d'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:54.635809 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 976/1934] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:49:54 2022] DELETE /identity/v3/users/d4d4dc994f9744ab990866ae8def199d => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:49:54.640691 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70be1a3-b401-4f22-bb88-7dc80daf6c41 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:54.641019 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70be1a3-b401-4f22-bb88-7dc80daf6c41 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:54.641300 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70be1a3-b401-4f22-bb88-7dc80daf6c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:54.663995 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-e70be1a3-b401-4f22-bb88-7dc80daf6c41 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:49:54.665365 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-e70be1a3-b401-4f22-bb88-7dc80daf6c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:54.758591 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 977/1935] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:49:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:49:54.768991 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 959/1936] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:49:54 2022] PUT /identity/v3/projects/c9acb65d67d74d5aa835a0840faf5336/users/167bf96ff5894771ab52d22ddb3c5b07/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:49:54.769479 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70be1a3-b401-4f22-bb88-7dc80daf6c41 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:54.771240 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-e70be1a3-b401-4f22-bb88-7dc80daf6c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:54.775681 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2c6947-114f-4401-9789-9bfb9b40dc37 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:54.786315 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-2d2c6947-114f-4401-9789-9bfb9b40dc37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:54.794843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:54.886977 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70be1a3-b401-4f22-bb88-7dc80daf6c41 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:54.928540 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2c6947-114f-4401-9789-9bfb9b40dc37 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:54.935330 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202bfea-9102-405e-b862-536cef5541df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:54.935967 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202bfea-9102-405e-b862-536cef5541df admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:54.936272 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202bfea-9102-405e-b862-536cef5541df admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:54.936515 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202bfea-9102-405e-b862-536cef5541df admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:54.937286 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202bfea-9102-405e-b862-536cef5541df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:54.938920 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202bfea-9102-405e-b862-536cef5541df admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:54.961873 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 960/1937] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:49:54 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:49:54.963202 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202bfea-9102-405e-b862-536cef5541df admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:54.968770 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8491ac-abee-43b1-858e-7b2913eaeaa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:54.969589 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8491ac-abee-43b1-858e-7b2913eaeaa5 service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:54.970009 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8491ac-abee-43b1-858e-7b2913eaeaa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:54.970429 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8491ac-abee-43b1-858e-7b2913eaeaa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:54.972528 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:54.978921 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202bfea-9102-405e-b862-536cef5541df admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:54.984195 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983e230b-c3e5-4795-8085-bd0c1ff38582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:54.984931 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e230b-c3e5-4795-8085-bd0c1ff38582 admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:54.985215 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e230b-c3e5-4795-8085-bd0c1ff38582 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:54.985494 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e230b-c3e5-4795-8085-bd0c1ff38582 admin admin] PATH_INFO: `/v3/projects/c9acb65d67d74d5aa835a0840faf5336/users/167bf96ff5894771ab52d22ddb3c5b07/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:54.986626 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8491ac-abee-43b1-858e-7b2913eaeaa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:54.988396 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983e230b-c3e5-4795-8085-bd0c1ff38582 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9acb65d67d74d5aa835a0840faf5336, user_id=167bf96ff5894771ab52d22ddb3c5b07, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:54.988874 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8491ac-abee-43b1-858e-7b2913eaeaa5 service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:54.991234 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983e230b-c3e5-4795-8085-bd0c1ff38582 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:54.999924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 978/1938] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:49:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:49:55.006820 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 961/1939] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:49:54 2022] PUT /identity/v3/projects/c9acb65d67d74d5aa835a0840faf5336/users/167bf96ff5894771ab52d22ddb3c5b07/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:49:55.019177 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a07d56-cb78-47b6-801e-6ab8ee70ed30 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:55.019532 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a07d56-cb78-47b6-801e-6ab8ee70ed30 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:55.019805 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a07d56-cb78-47b6-801e-6ab8ee70ed30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:55.062690 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-33a07d56-cb78-47b6-801e-6ab8ee70ed30 None None] MFA Rules not processed for user `167bf96ff5894771ab52d22ddb3c5b07`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:49:55.064464 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-33a07d56-cb78-47b6-801e-6ab8ee70ed30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:55.152736 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 979/1940] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:49:55 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:49:55.165616 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983e230b-c3e5-4795-8085-bd0c1ff38582 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:55.167305 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-983e230b-c3e5-4795-8085-bd0c1ff38582 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:55.229560 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983e230b-c3e5-4795-8085-bd0c1ff38582 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:55.233211 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da96d7ef-26ff-4dcb-976a-205181a551a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:55.233820 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96d7ef-26ff-4dcb-976a-205181a551a9 service cinder] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:55.234193 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96d7ef-26ff-4dcb-976a-205181a551a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:55.234435 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96d7ef-26ff-4dcb-976a-205181a551a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:55.244638 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da96d7ef-26ff-4dcb-976a-205181a551a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:55.246114 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da96d7ef-26ff-4dcb-976a-205181a551a9 service cinder] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:55.256977 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 962/1941] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:49:55 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:49:55.371277 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a07d56-cb78-47b6-801e-6ab8ee70ed30 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:55.372743 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-33a07d56-cb78-47b6-801e-6ab8ee70ed30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:55.388600 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a07d56-cb78-47b6-801e-6ab8ee70ed30 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:55.395649 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e10e11-7cb2-49ce-9954-a9669e2142bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:55.396304 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e10e11-7cb2-49ce-9954-a9669e2142bc service cinder] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:55.396606 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e10e11-7cb2-49ce-9954-a9669e2142bc service cinder] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:55.397024 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e10e11-7cb2-49ce-9954-a9669e2142bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:55.398868 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-14e10e11-7cb2-49ce-9954-a9669e2142bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:55.462337 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:55.466135 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e10e11-7cb2-49ce-9954-a9669e2142bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:55.467685 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e10e11-7cb2-49ce-9954-a9669e2142bc service cinder] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:55.476816 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 980/1942] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:49:55 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:49:55.538966 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da96d7ef-26ff-4dcb-976a-205181a551a9 service cinder] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:55.540864 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-da96d7ef-26ff-4dcb-976a-205181a551a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:55.628643 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da96d7ef-26ff-4dcb-976a-205181a551a9 service cinder] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:55.633975 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5918159-0153-4e3a-8d7b-c8940292cb7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:55.634605 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5918159-0153-4e3a-8d7b-c8940292cb7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:55.635456 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5918159-0153-4e3a-8d7b-c8940292cb7b admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:55.635738 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5918159-0153-4e3a-8d7b-c8940292cb7b admin admin] PATH_INFO: `/v3/projects/a9eeacff0e01442c9a739b43590284d0` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:55.637045 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5918159-0153-4e3a-8d7b-c8940292cb7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9eeacff0e01442c9a739b43590284d0)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:55.639586 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5918159-0153-4e3a-8d7b-c8940292cb7b admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:55.742173 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-b5918159-0153-4e3a-8d7b-c8940292cb7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9eeacff0e01442c9a739b43590284d0', 'request_id': 'req-b5918159-0153-4e3a-8d7b-c8940292cb7b'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:55.801783 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-b5918159-0153-4e3a-8d7b-c8940292cb7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9eeacff0e01442c9a739b43590284d0', 'request_id': 'req-b5918159-0153-4e3a-8d7b-c8940292cb7b'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:55.807758 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-b5918159-0153-4e3a-8d7b-c8940292cb7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9eeacff0e01442c9a739b43590284d0', 'request_id': 'req-b5918159-0153-4e3a-8d7b-c8940292cb7b'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:55.820798 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-b5918159-0153-4e3a-8d7b-c8940292cb7b admin admin] The token cache is being invalidate because project a9eeacff0e01442c9a739b43590284d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:49:55.821095 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-b5918159-0153-4e3a-8d7b-c8940292cb7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:55.821825 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-b5918159-0153-4e3a-8d7b-c8940292cb7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:55.823384 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 963/1943] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:49:55 2022] DELETE /identity/v3/projects/a9eeacff0e01442c9a739b43590284d0 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:49:55.829810 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd133ce-4620-4fef-833e-411fe4f8305e None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:55.830098 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd133ce-4620-4fef-833e-411fe4f8305e None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:55.830377 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd133ce-4620-4fef-833e-411fe4f8305e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:55.850644 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-9dd133ce-4620-4fef-833e-411fe4f8305e None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:49:55.851968 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-9dd133ce-4620-4fef-833e-411fe4f8305e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:55.915034 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 981/1944] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:49:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:49:55.923097 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be011d9-80e4-499e-9d6e-957699eb8f72 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:55.923385 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be011d9-80e4-499e-9d6e-957699eb8f72 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:55.923664 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be011d9-80e4-499e-9d6e-957699eb8f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:55.944105 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-4be011d9-80e4-499e-9d6e-957699eb8f72 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:49:55.945395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-4be011d9-80e4-499e-9d6e-957699eb8f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:55.953229 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 964/1945] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:49:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:49:55.958199 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd133ce-4620-4fef-833e-411fe4f8305e None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:55.967823 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd133ce-4620-4fef-833e-411fe4f8305e None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:55.973284 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80934cc-a4e9-448c-b7ad-a952b53ef88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:55.973888 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80934cc-a4e9-448c-b7ad-a952b53ef88a admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:55.974168 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80934cc-a4e9-448c-b7ad-a952b53ef88a admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:55.974445 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80934cc-a4e9-448c-b7ad-a952b53ef88a admin admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:55.975094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80934cc-a4e9-448c-b7ad-a952b53ef88a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:49:55.975400 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80934cc-a4e9-448c-b7ad-a952b53ef88a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:55.977029 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80934cc-a4e9-448c-b7ad-a952b53ef88a admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:55.990130 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 982/1946] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:49:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:49:55.994605 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be011d9-80e4-499e-9d6e-957699eb8f72 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:56.006000 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be011d9-80e4-499e-9d6e-957699eb8f72 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:56.010826 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e757aaa7-ead3-430c-baf6-175720230d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:56.011441 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757aaa7-ead3-430c-baf6-175720230d38 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:56.011708 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757aaa7-ead3-430c-baf6-175720230d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:56.012044 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757aaa7-ead3-430c-baf6-175720230d38 admin admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:56.012650 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e757aaa7-ead3-430c-baf6-175720230d38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:56.014592 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e757aaa7-ead3-430c-baf6-175720230d38 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:56.032392 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:56.101516 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 965/1947] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:49:55 2022] POST /identity/v3/projects => generated 399 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:49:56.105481 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80934cc-a4e9-448c-b7ad-a952b53ef88a admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:56.116930 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80934cc-a4e9-448c-b7ad-a952b53ef88a admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:56.195588 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900ad6a-87e0-4839-b1d7-9489786b1ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:56.196235 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900ad6a-87e0-4839-b1d7-9489786b1ace admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:56.196511 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900ad6a-87e0-4839-b1d7-9489786b1ace admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:56.196826 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900ad6a-87e0-4839-b1d7-9489786b1ace admin admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:56.197473 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b900ad6a-87e0-4839-b1d7-9489786b1ace admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:56.199569 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b900ad6a-87e0-4839-b1d7-9489786b1ace admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:56.234048 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e757aaa7-ead3-430c-baf6-175720230d38 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:56.236135 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-e757aaa7-ead3-430c-baf6-175720230d38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:56.250125 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e757aaa7-ead3-430c-baf6-175720230d38 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:56.256132 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f59ef-104e-4a7c-8f59-0eefee5671a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:56.256975 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f59ef-104e-4a7c-8f59-0eefee5671a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:56.257262 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f59ef-104e-4a7c-8f59-0eefee5671a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:56.257544 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f59ef-104e-4a7c-8f59-0eefee5671a3 admin admin] PATH_INFO: `/v3/users/fe7c89dca78f4569ba1c144026ca0a95` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:56.258804 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800f59ef-104e-4a7c-8f59-0eefee5671a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe7c89dca78f4569ba1c144026ca0a95)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:56.261055 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800f59ef-104e-4a7c-8f59-0eefee5671a3 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:56.316938 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 983/1948] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:49:56 2022] POST /identity/v3/users => generated 300 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:49:56.318111 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900ad6a-87e0-4839-b1d7-9489786b1ace admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:56.328639 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900ad6a-87e0-4839-b1d7-9489786b1ace admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:56.335631 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122afc08-69f5-4c90-b92a-153774462e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:56.336233 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122afc08-69f5-4c90-b92a-153774462e6d admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:56.336510 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122afc08-69f5-4c90-b92a-153774462e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:56.336842 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122afc08-69f5-4c90-b92a-153774462e6d admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:56.337499 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122afc08-69f5-4c90-b92a-153774462e6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:56.339232 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122afc08-69f5-4c90-b92a-153774462e6d admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:56.351918 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 984/1949] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:49:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:49:56.356095 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122afc08-69f5-4c90-b92a-153774462e6d admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:56.369262 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122afc08-69f5-4c90-b92a-153774462e6d admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:56.371804 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eb2563-5c79-490c-afec-48918c934b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:56.372414 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb2563-5c79-490c-afec-48918c934b06 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:56.372762 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb2563-5c79-490c-afec-48918c934b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:56.373041 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb2563-5c79-490c-afec-48918c934b06 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:56.373731 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eb2563-5c79-490c-afec-48918c934b06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:56.375339 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eb2563-5c79-490c-afec-48918c934b06 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:56.386195 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 985/1950] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:49:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:49:56.390601 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eb2563-5c79-490c-afec-48918c934b06 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:56.403580 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eb2563-5c79-490c-afec-48918c934b06 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:56.408261 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e553a6a-f99d-46d8-82c4-14935eed1e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:56.410306 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e553a6a-f99d-46d8-82c4-14935eed1e74 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:56.410625 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e553a6a-f99d-46d8-82c4-14935eed1e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:56.411038 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e553a6a-f99d-46d8-82c4-14935eed1e74 admin admin] PATH_INFO: `/v3/projects/4ec2507572bf4ecba09f797183706e5a/users/82831a7fb04e4a94b767e957b4574897/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:56.427902 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e553a6a-f99d-46d8-82c4-14935eed1e74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ec2507572bf4ecba09f797183706e5a, user_id=82831a7fb04e4a94b767e957b4574897, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:56.430662 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e553a6a-f99d-46d8-82c4-14935eed1e74 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:56.545779 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 986/1951] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:49:56 2022] PUT /identity/v3/projects/4ec2507572bf4ecba09f797183706e5a/users/82831a7fb04e4a94b767e957b4574897/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:49:56.551628 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-800f59ef-104e-4a7c-8f59-0eefee5671a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe7c89dca78f4569ba1c144026ca0a95'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:56.556556 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5dfd17-cbed-4023-b0a7-06ce71a585e9 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:56.556828 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5dfd17-cbed-4023-b0a7-06ce71a585e9 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:56.557109 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5dfd17-cbed-4023-b0a7-06ce71a585e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:56.574539 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-800f59ef-104e-4a7c-8f59-0eefee5671a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe7c89dca78f4569ba1c144026ca0a95'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:56.585113 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-800f59ef-104e-4a7c-8f59-0eefee5671a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe7c89dca78f4569ba1c144026ca0a95'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:56.611384 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-ae5dfd17-cbed-4023-b0a7-06ce71a585e9 None None] MFA Rules not processed for user `82831a7fb04e4a94b767e957b4574897`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:49:56.611384 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-ae5dfd17-cbed-4023-b0a7-06ce71a585e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:56.620394 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:56.675089 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 987/1952] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:49:56 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:49:56.691099 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5dfd17-cbed-4023-b0a7-06ce71a585e9 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:56.693244 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-ae5dfd17-cbed-4023-b0a7-06ce71a585e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:56.693852 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-800f59ef-104e-4a7c-8f59-0eefee5671a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe7c89dca78f4569ba1c144026ca0a95'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:56.697713 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-800f59ef-104e-4a7c-8f59-0eefee5671a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe7c89dca78f4569ba1c144026ca0a95'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:56.701119 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 966/1953] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:49:56 2022] DELETE /identity/v3/users/fe7c89dca78f4569ba1c144026ca0a95 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:49:56.706701 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac881f6-2dd5-4066-8220-a86b813be45b None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:56.707029 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac881f6-2dd5-4066-8220-a86b813be45b None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:56.707359 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac881f6-2dd5-4066-8220-a86b813be45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:56.727603 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-3ac881f6-2dd5-4066-8220-a86b813be45b None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:49:56.728860 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-3ac881f6-2dd5-4066-8220-a86b813be45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:56.753453 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5dfd17-cbed-4023-b0a7-06ce71a585e9 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:56.791926 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 967/1954] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:49:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:49:56.803147 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac881f6-2dd5-4066-8220-a86b813be45b None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:56.804493 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-3ac881f6-2dd5-4066-8220-a86b813be45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:56.805346 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07db8c5-77ed-4d6e-86a8-440a0fa2842f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:56.805955 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07db8c5-77ed-4d6e-86a8-440a0fa2842f service cinder] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:56.806253 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07db8c5-77ed-4d6e-86a8-440a0fa2842f service cinder] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:56.806525 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07db8c5-77ed-4d6e-86a8-440a0fa2842f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:56.808319 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-c07db8c5-77ed-4d6e-86a8-440a0fa2842f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:56.865931 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac881f6-2dd5-4066-8220-a86b813be45b None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:56.869350 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07db8c5-77ed-4d6e-86a8-440a0fa2842f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:56.869894 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3a80e-1b58-46ee-abbc-b88ac9ac9ca7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:56.870533 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3a80e-1b58-46ee-abbc-b88ac9ac9ca7 service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:56.870835 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3a80e-1b58-46ee-abbc-b88ac9ac9ca7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:56.871149 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3a80e-1b58-46ee-abbc-b88ac9ac9ca7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:56.871303 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07db8c5-77ed-4d6e-86a8-440a0fa2842f service cinder] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:56.881133 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac3a80e-1b58-46ee-abbc-b88ac9ac9ca7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:56.881656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 988/1955] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:49:56 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:49:56.882724 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac3a80e-1b58-46ee-abbc-b88ac9ac9ca7 service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:56.892663 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 968/1956] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:49:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:49:57.359279 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07db8c5-77ed-4d6e-86a8-440a0fa2842f service cinder] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:57.360831 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-c07db8c5-77ed-4d6e-86a8-440a0fa2842f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:57.377375 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07db8c5-77ed-4d6e-86a8-440a0fa2842f service cinder] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:57.384663 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c0f1f1-6039-4bba-9aa4-e22307b018d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:57.385574 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0f1f1-6039-4bba-9aa4-e22307b018d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:57.386048 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0f1f1-6039-4bba-9aa4-e22307b018d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:57.386527 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0f1f1-6039-4bba-9aa4-e22307b018d8 admin admin] PATH_INFO: `/v3/projects/77d0588c9a2a4bc9b1fac934c2ca7f70` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:57.387773 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c0f1f1-6039-4bba-9aa4-e22307b018d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77d0588c9a2a4bc9b1fac934c2ca7f70)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:57.390087 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c0f1f1-6039-4bba-9aa4-e22307b018d8 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:57.414944 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:57.499347 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-76c0f1f1-6039-4bba-9aa4-e22307b018d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77d0588c9a2a4bc9b1fac934c2ca7f70', 'request_id': 'req-76c0f1f1-6039-4bba-9aa4-e22307b018d8'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:57.506526 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-76c0f1f1-6039-4bba-9aa4-e22307b018d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77d0588c9a2a4bc9b1fac934c2ca7f70', 'request_id': 'req-76c0f1f1-6039-4bba-9aa4-e22307b018d8'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:57.513536 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-76c0f1f1-6039-4bba-9aa4-e22307b018d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77d0588c9a2a4bc9b1fac934c2ca7f70', 'request_id': 'req-76c0f1f1-6039-4bba-9aa4-e22307b018d8'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:57.649704 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-76c0f1f1-6039-4bba-9aa4-e22307b018d8 admin admin] The token cache is being invalidate because project 77d0588c9a2a4bc9b1fac934c2ca7f70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:49:57.650174 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-76c0f1f1-6039-4bba-9aa4-e22307b018d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:57.651158 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-76c0f1f1-6039-4bba-9aa4-e22307b018d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:57.653205 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 989/1957] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:49:57 2022] DELETE /identity/v3/projects/77d0588c9a2a4bc9b1fac934c2ca7f70 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:49:57.656986 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3a80e-1b58-46ee-abbc-b88ac9ac9ca7 service neutron] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:57.667162 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3a80e-1b58-46ee-abbc-b88ac9ac9ca7 service neutron] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:57.678425 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:57.732088 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7243b090-4ed1-451a-8f4e-72672697bc47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:57.732893 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243b090-4ed1-451a-8f4e-72672697bc47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:57.733335 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243b090-4ed1-451a-8f4e-72672697bc47 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:57.733677 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243b090-4ed1-451a-8f4e-72672697bc47 admin admin] PATH_INFO: `/v3/users/18ed6518c55f4bc293f440ead11aae5f` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:57.734824 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7243b090-4ed1-451a-8f4e-72672697bc47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18ed6518c55f4bc293f440ead11aae5f)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:57.737863 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7243b090-4ed1-451a-8f4e-72672697bc47 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:57.973517 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-7243b090-4ed1-451a-8f4e-72672697bc47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18ed6518c55f4bc293f440ead11aae5f'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:57.989153 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-7243b090-4ed1-451a-8f4e-72672697bc47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18ed6518c55f4bc293f440ead11aae5f'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:57.993533 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-7243b090-4ed1-451a-8f4e-72672697bc47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18ed6518c55f4bc293f440ead11aae5f'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:58.070631 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-7243b090-4ed1-451a-8f4e-72672697bc47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18ed6518c55f4bc293f440ead11aae5f'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:58.071401 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-7243b090-4ed1-451a-8f4e-72672697bc47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18ed6518c55f4bc293f440ead11aae5f'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:58.078308 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 969/1958] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:49:57 2022] DELETE /identity/v3/users/18ed6518c55f4bc293f440ead11aae5f => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:49:58.545751 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c0f1f1-6039-4bba-9aa4-e22307b018d8 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:58.552134 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-76c0f1f1-6039-4bba-9aa4-e22307b018d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:58.568561 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c0f1f1-6039-4bba-9aa4-e22307b018d8 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:58.637891 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bbf946-f1f7-44ec-ac83-91a8bd718652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:58.638618 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bbf946-f1f7-44ec-ac83-91a8bd718652 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:58.638989 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bbf946-f1f7-44ec-ac83-91a8bd718652 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:58.639306 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bbf946-f1f7-44ec-ac83-91a8bd718652 admin admin] PATH_INFO: `/v3/projects/83a2279ef5db4931af661ed7ce43688b` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:58.640408 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bbf946-f1f7-44ec-ac83-91a8bd718652 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83a2279ef5db4931af661ed7ce43688b)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:58.642670 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bbf946-f1f7-44ec-ac83-91a8bd718652 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:58.820387 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-09bbf946-f1f7-44ec-ac83-91a8bd718652 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83a2279ef5db4931af661ed7ce43688b', 'request_id': 'req-09bbf946-f1f7-44ec-ac83-91a8bd718652'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:58.833519 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-09bbf946-f1f7-44ec-ac83-91a8bd718652 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83a2279ef5db4931af661ed7ce43688b', 'request_id': 'req-09bbf946-f1f7-44ec-ac83-91a8bd718652'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:58.836587 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-09bbf946-f1f7-44ec-ac83-91a8bd718652 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83a2279ef5db4931af661ed7ce43688b', 'request_id': 'req-09bbf946-f1f7-44ec-ac83-91a8bd718652'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:58.965384 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-09bbf946-f1f7-44ec-ac83-91a8bd718652 admin admin] The token cache is being invalidate because project 83a2279ef5db4931af661ed7ce43688b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:49:58.965680 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-09bbf946-f1f7-44ec-ac83-91a8bd718652 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:58.966355 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-09bbf946-f1f7-44ec-ac83-91a8bd718652 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:49:58.968171 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 990/1959] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:49:58 2022] DELETE /identity/v3/projects/83a2279ef5db4931af661ed7ce43688b => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:49:58.973867 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d134990b-476c-4408-8641-0f835c7db544 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:58.974280 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d134990b-476c-4408-8641-0f835c7db544 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:58.974440 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d134990b-476c-4408-8641-0f835c7db544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:59.025147 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-d134990b-476c-4408-8641-0f835c7db544 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:49:59.026584 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-d134990b-476c-4408-8641-0f835c7db544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:59.097338 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 970/1960] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:49:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:49:59.112021 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cf912-e1fd-4536-8e6e-2a5acf04e50f None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:59.112021 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cf912-e1fd-4536-8e6e-2a5acf04e50f None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:59.112021 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cf912-e1fd-4536-8e6e-2a5acf04e50f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:59.125805 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-423cf912-e1fd-4536-8e6e-2a5acf04e50f None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:49:59.127014 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-423cf912-e1fd-4536-8e6e-2a5acf04e50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:59.136170 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 991/1961] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:49:59.141378 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d134990b-476c-4408-8641-0f835c7db544 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:59.152382 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d134990b-476c-4408-8641-0f835c7db544 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:59.157116 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25faf0a0-d6f2-41f8-8a32-0450a5b0658e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:59.157742 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25faf0a0-d6f2-41f8-8a32-0450a5b0658e admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:59.158035 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25faf0a0-d6f2-41f8-8a32-0450a5b0658e admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:59.158291 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25faf0a0-d6f2-41f8-8a32-0450a5b0658e admin admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:59.158996 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25faf0a0-d6f2-41f8-8a32-0450a5b0658e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:49:59.159367 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25faf0a0-d6f2-41f8-8a32-0450a5b0658e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:59.161002 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25faf0a0-d6f2-41f8-8a32-0450a5b0658e admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:59.181088 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 971/1962] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:49:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:49:59.182770 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423cf912-e1fd-4536-8e6e-2a5acf04e50f None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:59.194147 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423cf912-e1fd-4536-8e6e-2a5acf04e50f None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:59.199186 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc726e-302d-40bd-8241-d4f70f0e50c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:59.199797 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc726e-302d-40bd-8241-d4f70f0e50c1 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:59.200083 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc726e-302d-40bd-8241-d4f70f0e50c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:59.200359 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc726e-302d-40bd-8241-d4f70f0e50c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:59.201106 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cc726e-302d-40bd-8241-d4f70f0e50c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:59.203118 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cc726e-302d-40bd-8241-d4f70f0e50c1 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:59.324156 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 992/1963] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:49:59 2022] POST /identity/v3/projects => generated 385 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:49:59.328093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25faf0a0-d6f2-41f8-8a32-0450a5b0658e admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:59.338460 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25faf0a0-d6f2-41f8-8a32-0450a5b0658e admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:59.350463 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:59.399974 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5b978b-dbf5-45f8-8b2d-f586bf96d43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:59.400580 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b978b-dbf5-45f8-8b2d-f586bf96d43f admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:59.400897 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b978b-dbf5-45f8-8b2d-f586bf96d43f admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:59.401195 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b978b-dbf5-45f8-8b2d-f586bf96d43f admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:59.401847 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5b978b-dbf5-45f8-8b2d-f586bf96d43f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:59.403760 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5b978b-dbf5-45f8-8b2d-f586bf96d43f admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:59.558216 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 972/1964] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:49:59 2022] POST /identity/v3/users => generated 293 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:49:59.562306 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc726e-302d-40bd-8241-d4f70f0e50c1 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:59.572878 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc726e-302d-40bd-8241-d4f70f0e50c1 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:59.579826 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ff9a9-7dd2-4c11-8f08-5f7fbe6349fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:59.579826 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ff9a9-7dd2-4c11-8f08-5f7fbe6349fb admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:59.579826 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ff9a9-7dd2-4c11-8f08-5f7fbe6349fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:59.579826 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ff9a9-7dd2-4c11-8f08-5f7fbe6349fb admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:59.581342 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ff9a9-7dd2-4c11-8f08-5f7fbe6349fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:59.582799 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ff9a9-7dd2-4c11-8f08-5f7fbe6349fb admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:59.590818 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:59.612875 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 993/1965] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:49:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:49:59.616147 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5b978b-dbf5-45f8-8b2d-f586bf96d43f admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:59.628641 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5b978b-dbf5-45f8-8b2d-f586bf96d43f admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:59.634502 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9aba96-0c9a-4cd1-8be9-6305315ad44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:59.635113 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9aba96-0c9a-4cd1-8be9-6305315ad44a admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:59.635398 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9aba96-0c9a-4cd1-8be9-6305315ad44a admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:59.635672 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9aba96-0c9a-4cd1-8be9-6305315ad44a admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:59.636323 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9aba96-0c9a-4cd1-8be9-6305315ad44a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:59.637968 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9aba96-0c9a-4cd1-8be9-6305315ad44a admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:59.655220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 973/1966] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:49:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:49:59.656744 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ff9a9-7dd2-4c11-8f08-5f7fbe6349fb admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:59.669166 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ff9a9-7dd2-4c11-8f08-5f7fbe6349fb admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:59.675361 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8df0be-fa3a-4562-a238-f0a8cdb59429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:59.675361 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8df0be-fa3a-4562-a238-f0a8cdb59429 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:59.675361 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8df0be-fa3a-4562-a238-f0a8cdb59429 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:59.675640 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8df0be-fa3a-4562-a238-f0a8cdb59429 admin admin] PATH_INFO: `/v3/projects/0d3463b978574c838892dbb1207531d1/users/5f0a424074884916a5844e6f5d74580c/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:59.691276 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8df0be-fa3a-4562-a238-f0a8cdb59429 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d3463b978574c838892dbb1207531d1, user_id=5f0a424074884916a5844e6f5d74580c, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:59.694094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8df0be-fa3a-4562-a238-f0a8cdb59429 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:59.827186 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 994/1967] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:49:59 2022] PUT /identity/v3/projects/0d3463b978574c838892dbb1207531d1/users/5f0a424074884916a5844e6f5d74580c/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:49:59.844453 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c207176-2bfd-4741-b0f0-048f035d1f75 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:59.844453 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c207176-2bfd-4741-b0f0-048f035d1f75 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:59.844453 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c207176-2bfd-4741-b0f0-048f035d1f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:59.870849 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-5c207176-2bfd-4741-b0f0-048f035d1f75 None None] MFA Rules not processed for user `5f0a424074884916a5844e6f5d74580c`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:49:59.872048 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-5c207176-2bfd-4741-b0f0-048f035d1f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:59.937887 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 974/1968] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:49:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:49:59.943211 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8df0be-fa3a-4562-a238-f0a8cdb59429 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:00.012389 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8df0be-fa3a-4562-a238-f0a8cdb59429 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:00.027416 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f024e-6148-453a-8b12-3f8d7a94055b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:00.027416 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f024e-6148-453a-8b12-3f8d7a94055b admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:00.027416 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f024e-6148-453a-8b12-3f8d7a94055b admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:00.027416 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f024e-6148-453a-8b12-3f8d7a94055b admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:00.028220 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f024e-6148-453a-8b12-3f8d7a94055b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:00.028220 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f024e-6148-453a-8b12-3f8d7a94055b admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:00.035316 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c207176-2bfd-4741-b0f0-048f035d1f75 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:00.037259 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-5c207176-2bfd-4741-b0f0-048f035d1f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:00.097095 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c207176-2bfd-4741-b0f0-048f035d1f75 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:00.101067 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5fe2e8-0aff-4f80-9909-14d95f9aefc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:00.101725 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5fe2e8-0aff-4f80-9909-14d95f9aefc1 service cinder] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:00.102003 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5fe2e8-0aff-4f80-9909-14d95f9aefc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:00.102271 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5fe2e8-0aff-4f80-9909-14d95f9aefc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:00.104000 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-6c5fe2e8-0aff-4f80-9909-14d95f9aefc1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:00.119662 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 995/1969] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:49:59 2022] POST /identity/v3/projects => generated 387 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:50:00.122819 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f024e-6148-453a-8b12-3f8d7a94055b admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:00.179701 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:00.209591 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f024e-6148-453a-8b12-3f8d7a94055b admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:00.214544 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc726f-f818-4c25-97ca-94d2f44ef60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:00.215163 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc726f-f818-4c25-97ca-94d2f44ef60c admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:00.215763 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc726f-f818-4c25-97ca-94d2f44ef60c admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:00.216047 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc726f-f818-4c25-97ca-94d2f44ef60c admin admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:00.216526 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5fe2e8-0aff-4f80-9909-14d95f9aefc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:00.216706 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc726f-f818-4c25-97ca-94d2f44ef60c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:00.218001 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5fe2e8-0aff-4f80-9909-14d95f9aefc1 service cinder] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:00.218654 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc726f-f818-4c25-97ca-94d2f44ef60c admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:00.229324 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 975/1970] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:50:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:50:00.319590 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 996/1971] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:50:00 2022] POST /identity/v3/users => generated 300 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:50:00.323826 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5fe2e8-0aff-4f80-9909-14d95f9aefc1 service cinder] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:00.336538 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5fe2e8-0aff-4f80-9909-14d95f9aefc1 service cinder] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:00.341755 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2471f67b-791e-4e3b-b37c-de74f5d60374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:00.342369 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2471f67b-791e-4e3b-b37c-de74f5d60374 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:00.342654 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2471f67b-791e-4e3b-b37c-de74f5d60374 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:00.342928 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2471f67b-791e-4e3b-b37c-de74f5d60374 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:00.343679 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2471f67b-791e-4e3b-b37c-de74f5d60374 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:00.345588 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2471f67b-791e-4e3b-b37c-de74f5d60374 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:00.356707 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 976/1972] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:50:00.361133 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc726f-f818-4c25-97ca-94d2f44ef60c admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:00.371218 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc726f-f818-4c25-97ca-94d2f44ef60c admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:00.375882 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6dea29-66ee-40af-9ea1-470a244fe2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:00.376595 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6dea29-66ee-40af-9ea1-470a244fe2ed admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:00.376914 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6dea29-66ee-40af-9ea1-470a244fe2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:00.380742 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6dea29-66ee-40af-9ea1-470a244fe2ed admin admin] PATH_INFO: `/v3/projects/0869079882384b48b8ff7a85c1b9da8f/users/7d4b373bd5c04262b5124054f1c063b8/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:00.394920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6dea29-66ee-40af-9ea1-470a244fe2ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=0869079882384b48b8ff7a85c1b9da8f, user_id=7d4b373bd5c04262b5124054f1c063b8, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:00.398309 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6dea29-66ee-40af-9ea1-470a244fe2ed admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:00.493396 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 997/1973] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:50:00 2022] PUT /identity/v3/projects/0869079882384b48b8ff7a85c1b9da8f/users/7d4b373bd5c04262b5124054f1c063b8/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:00.497748 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2471f67b-791e-4e3b-b37c-de74f5d60374 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:00.508659 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2471f67b-791e-4e3b-b37c-de74f5d60374 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:00.568645 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad89d5d-af72-40a1-b6cc-ac7fff4f123b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:00.569494 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad89d5d-af72-40a1-b6cc-ac7fff4f123b admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:00.570154 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad89d5d-af72-40a1-b6cc-ac7fff4f123b admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:00.570802 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad89d5d-af72-40a1-b6cc-ac7fff4f123b admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:00.571448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad89d5d-af72-40a1-b6cc-ac7fff4f123b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:00.573069 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad89d5d-af72-40a1-b6cc-ac7fff4f123b admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:00.583993 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 977/1974] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:00 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:50:00.588306 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6dea29-66ee-40af-9ea1-470a244fe2ed admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:00.599261 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6dea29-66ee-40af-9ea1-470a244fe2ed admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:00.604770 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1167a08a-da2f-470f-926a-71993129a2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:00.605495 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167a08a-da2f-470f-926a-71993129a2b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:00.605779 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167a08a-da2f-470f-926a-71993129a2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:00.606050 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167a08a-da2f-470f-926a-71993129a2b5 admin admin] PATH_INFO: `/v3/projects/0869079882384b48b8ff7a85c1b9da8f/users/7d4b373bd5c04262b5124054f1c063b8/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:00.608182 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1167a08a-da2f-470f-926a-71993129a2b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0869079882384b48b8ff7a85c1b9da8f, user_id=7d4b373bd5c04262b5124054f1c063b8, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:00.610922 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1167a08a-da2f-470f-926a-71993129a2b5 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:00.635571 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 998/1975] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:50:00 2022] PUT /identity/v3/projects/0869079882384b48b8ff7a85c1b9da8f/users/7d4b373bd5c04262b5124054f1c063b8/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:00.637751 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad40f3b-3cfc-4c1f-9a7a-c34157083a19 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:00.638040 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad40f3b-3cfc-4c1f-9a7a-c34157083a19 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:00.638318 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad40f3b-3cfc-4c1f-9a7a-c34157083a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:00.668707 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-fad40f3b-3cfc-4c1f-9a7a-c34157083a19 None None] MFA Rules not processed for user `7d4b373bd5c04262b5124054f1c063b8`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:00.670425 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-fad40f3b-3cfc-4c1f-9a7a-c34157083a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:00.749972 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 978/1976] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:00 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:50:01.449137 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1167a08a-da2f-470f-926a-71993129a2b5 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:01.451022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-1167a08a-da2f-470f-926a-71993129a2b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:01.467064 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:01.511074 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1167a08a-da2f-470f-926a-71993129a2b5 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:01.514748 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1027a4-8923-4425-929d-f04cd67cb6c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:01.515389 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1027a4-8923-4425-929d-f04cd67cb6c2 service cinder] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:01.515671 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1027a4-8923-4425-929d-f04cd67cb6c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:01.515978 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1027a4-8923-4425-929d-f04cd67cb6c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:01.531196 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1027a4-8923-4425-929d-f04cd67cb6c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:01.537455 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1027a4-8923-4425-929d-f04cd67cb6c2 service cinder] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:01.553025 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 999/1977] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:50:01 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:50:02.920105 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad40f3b-3cfc-4c1f-9a7a-c34157083a19 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:02.921581 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-fad40f3b-3cfc-4c1f-9a7a-c34157083a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:03.001209 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:03.015590 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad40f3b-3cfc-4c1f-9a7a-c34157083a19 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:03.021713 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7511c539-3c6d-4a65-8c97-9a4949adb45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:03.022346 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511c539-3c6d-4a65-8c97-9a4949adb45a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:03.022627 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511c539-3c6d-4a65-8c97-9a4949adb45a admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:03.022903 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511c539-3c6d-4a65-8c97-9a4949adb45a admin admin] PATH_INFO: `/v3/users/26c148d4a1fc40969d0a29cb8aea1a56` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:03.024014 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7511c539-3c6d-4a65-8c97-9a4949adb45a admin admin] RBAC: Authorizing `identity:delete_user(user_id=26c148d4a1fc40969d0a29cb8aea1a56)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:03.025976 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7511c539-3c6d-4a65-8c97-9a4949adb45a admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:03.260569 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-7511c539-3c6d-4a65-8c97-9a4949adb45a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26c148d4a1fc40969d0a29cb8aea1a56'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:03.275953 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-7511c539-3c6d-4a65-8c97-9a4949adb45a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26c148d4a1fc40969d0a29cb8aea1a56'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:03.285821 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-7511c539-3c6d-4a65-8c97-9a4949adb45a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26c148d4a1fc40969d0a29cb8aea1a56'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:03.362806 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-7511c539-3c6d-4a65-8c97-9a4949adb45a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26c148d4a1fc40969d0a29cb8aea1a56'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:03.363918 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-7511c539-3c6d-4a65-8c97-9a4949adb45a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26c148d4a1fc40969d0a29cb8aea1a56'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:03.370245 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 979/1978] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:50:02 2022] DELETE /identity/v3/users/26c148d4a1fc40969d0a29cb8aea1a56 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:03.374749 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df3200-6762-4b0e-885b-c8c97380e503 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:03.375247 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df3200-6762-4b0e-885b-c8c97380e503 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:03.375338 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df3200-6762-4b0e-885b-c8c97380e503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:03.395073 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-52df3200-6762-4b0e-885b-c8c97380e503 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:03.396244 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-52df3200-6762-4b0e-885b-c8c97380e503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:03.464134 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1000/1979] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:50:03.474750 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7511c539-3c6d-4a65-8c97-9a4949adb45a admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:03.476208 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-7511c539-3c6d-4a65-8c97-9a4949adb45a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:03.534626 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7511c539-3c6d-4a65-8c97-9a4949adb45a admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:03.538470 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1365e-4250-40fb-b252-dc54a3e6ccfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:03.539077 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1365e-4250-40fb-b252-dc54a3e6ccfd service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:03.539988 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1365e-4250-40fb-b252-dc54a3e6ccfd service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:03.539988 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1365e-4250-40fb-b252-dc54a3e6ccfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:03.549381 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1365e-4250-40fb-b252-dc54a3e6ccfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:03.552279 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1365e-4250-40fb-b252-dc54a3e6ccfd service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:03.563602 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 980/1980] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:50:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:50:03.567881 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52df3200-6762-4b0e-885b-c8c97380e503 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:03.569378 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-52df3200-6762-4b0e-885b-c8c97380e503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:03.583404 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52df3200-6762-4b0e-885b-c8c97380e503 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:03.588078 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fcf987-ebcd-4122-9d47-5b95df2ed7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:03.588736 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fcf987-ebcd-4122-9d47-5b95df2ed7f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:03.589038 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fcf987-ebcd-4122-9d47-5b95df2ed7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:03.589316 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fcf987-ebcd-4122-9d47-5b95df2ed7f0 admin admin] PATH_INFO: `/v3/users/5f0a424074884916a5844e6f5d74580c` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:03.590491 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fcf987-ebcd-4122-9d47-5b95df2ed7f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f0a424074884916a5844e6f5d74580c)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:03.592421 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fcf987-ebcd-4122-9d47-5b95df2ed7f0 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:03.796373 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-19fcf987-ebcd-4122-9d47-5b95df2ed7f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f0a424074884916a5844e6f5d74580c'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:03.866420 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-19fcf987-ebcd-4122-9d47-5b95df2ed7f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f0a424074884916a5844e6f5d74580c'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:03.891450 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-19fcf987-ebcd-4122-9d47-5b95df2ed7f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f0a424074884916a5844e6f5d74580c'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:03.903824 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-19fcf987-ebcd-4122-9d47-5b95df2ed7f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f0a424074884916a5844e6f5d74580c'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:03.903824 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-19fcf987-ebcd-4122-9d47-5b95df2ed7f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f0a424074884916a5844e6f5d74580c'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:03.906440 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1001/1981] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:50:03 2022] DELETE /identity/v3/users/5f0a424074884916a5844e6f5d74580c => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:03.911184 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83749283-481d-4d89-88eb-d01e98fa82dc None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:03.911471 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83749283-481d-4d89-88eb-d01e98fa82dc None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:03.911742 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83749283-481d-4d89-88eb-d01e98fa82dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:03.931701 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-83749283-481d-4d89-88eb-d01e98fa82dc None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:03.932946 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-83749283-481d-4d89-88eb-d01e98fa82dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:03.993893 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 981/1982] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:50:04.003384 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fcf987-ebcd-4122-9d47-5b95df2ed7f0 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:04.005772 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-19fcf987-ebcd-4122-9d47-5b95df2ed7f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:04.037291 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:04.063851 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fcf987-ebcd-4122-9d47-5b95df2ed7f0 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:04.068691 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e823abb-890e-4817-a8e2-bd44bb848513 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:04.069333 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e823abb-890e-4817-a8e2-bd44bb848513 service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:04.069614 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e823abb-890e-4817-a8e2-bd44bb848513 service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:04.069889 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e823abb-890e-4817-a8e2-bd44bb848513 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:04.076231 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83749283-481d-4d89-88eb-d01e98fa82dc None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:04.076231 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-83749283-481d-4d89-88eb-d01e98fa82dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:04.081925 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e823abb-890e-4817-a8e2-bd44bb848513 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:04.083443 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e823abb-890e-4817-a8e2-bd44bb848513 service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:04.093054 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83749283-481d-4d89-88eb-d01e98fa82dc None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:04.096397 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1002/1983] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:50:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:50:04.099884 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d14da70-a226-422c-b555-f32520e12803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:04.100676 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d14da70-a226-422c-b555-f32520e12803 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:04.100862 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d14da70-a226-422c-b555-f32520e12803 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:04.101186 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d14da70-a226-422c-b555-f32520e12803 admin admin] PATH_INFO: `/v3/projects/109d626ecc0348769c7c3e8bfd0322d8` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:04.104293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d14da70-a226-422c-b555-f32520e12803 admin admin] RBAC: Authorizing `identity:delete_project(project_id=109d626ecc0348769c7c3e8bfd0322d8)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:04.106021 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d14da70-a226-422c-b555-f32520e12803 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:04.115571 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:04.231217 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-2d14da70-a226-422c-b555-f32520e12803 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '109d626ecc0348769c7c3e8bfd0322d8', 'request_id': 'req-2d14da70-a226-422c-b555-f32520e12803'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:04.315894 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-2d14da70-a226-422c-b555-f32520e12803 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '109d626ecc0348769c7c3e8bfd0322d8', 'request_id': 'req-2d14da70-a226-422c-b555-f32520e12803'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:04.323878 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-2d14da70-a226-422c-b555-f32520e12803 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '109d626ecc0348769c7c3e8bfd0322d8', 'request_id': 'req-2d14da70-a226-422c-b555-f32520e12803'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:04.338114 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-2d14da70-a226-422c-b555-f32520e12803 admin admin] The token cache is being invalidate because project 109d626ecc0348769c7c3e8bfd0322d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:50:04.338425 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-2d14da70-a226-422c-b555-f32520e12803 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:04.339219 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-2d14da70-a226-422c-b555-f32520e12803 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:04.340889 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 982/1984] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:50:04 2022] DELETE /identity/v3/projects/109d626ecc0348769c7c3e8bfd0322d8 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:50:04.344970 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e823abb-890e-4817-a8e2-bd44bb848513 service neutron] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:04.359963 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e823abb-890e-4817-a8e2-bd44bb848513 service neutron] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:04.431203 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3f22cc-2935-4caa-9082-44c6fbf55657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:04.431969 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f22cc-2935-4caa-9082-44c6fbf55657 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:04.432257 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f22cc-2935-4caa-9082-44c6fbf55657 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:04.432529 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f22cc-2935-4caa-9082-44c6fbf55657 admin admin] PATH_INFO: `/v3/users/167bf96ff5894771ab52d22ddb3c5b07` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:04.433706 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3f22cc-2935-4caa-9082-44c6fbf55657 admin admin] RBAC: Authorizing `identity:delete_user(user_id=167bf96ff5894771ab52d22ddb3c5b07)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:04.435628 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3f22cc-2935-4caa-9082-44c6fbf55657 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:04.570633 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d14da70-a226-422c-b555-f32520e12803 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:04.572132 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-2d14da70-a226-422c-b555-f32520e12803 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:04.587148 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d14da70-a226-422c-b555-f32520e12803 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:04.592294 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0002604b-3f2d-4625-b59a-107972ae91b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:04.593048 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0002604b-3f2d-4625-b59a-107972ae91b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:04.593278 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0002604b-3f2d-4625-b59a-107972ae91b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:04.593642 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0002604b-3f2d-4625-b59a-107972ae91b2 admin admin] PATH_INFO: `/v3/projects/0d3463b978574c838892dbb1207531d1` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:04.595439 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0002604b-3f2d-4625-b59a-107972ae91b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d3463b978574c838892dbb1207531d1)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:04.598661 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0002604b-3f2d-4625-b59a-107972ae91b2 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:04.651853 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-7a3f22cc-2935-4caa-9082-44c6fbf55657 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '167bf96ff5894771ab52d22ddb3c5b07'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:04.724182 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-7a3f22cc-2935-4caa-9082-44c6fbf55657 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '167bf96ff5894771ab52d22ddb3c5b07'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:04.725364 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-0002604b-3f2d-4625-b59a-107972ae91b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d3463b978574c838892dbb1207531d1', 'request_id': 'req-0002604b-3f2d-4625-b59a-107972ae91b2'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:04.737304 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-7a3f22cc-2935-4caa-9082-44c6fbf55657 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '167bf96ff5894771ab52d22ddb3c5b07'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:04.744437 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-7a3f22cc-2935-4caa-9082-44c6fbf55657 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '167bf96ff5894771ab52d22ddb3c5b07'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:04.745137 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-7a3f22cc-2935-4caa-9082-44c6fbf55657 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '167bf96ff5894771ab52d22ddb3c5b07'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:04.751106 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1003/1985] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:50:04 2022] DELETE /identity/v3/users/167bf96ff5894771ab52d22ddb3c5b07 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:04.854701 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-0002604b-3f2d-4625-b59a-107972ae91b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d3463b978574c838892dbb1207531d1', 'request_id': 'req-0002604b-3f2d-4625-b59a-107972ae91b2'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:04.860810 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-0002604b-3f2d-4625-b59a-107972ae91b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d3463b978574c838892dbb1207531d1', 'request_id': 'req-0002604b-3f2d-4625-b59a-107972ae91b2'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:04.872167 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-0002604b-3f2d-4625-b59a-107972ae91b2 admin admin] The token cache is being invalidate because project 0d3463b978574c838892dbb1207531d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:50:04.872454 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-0002604b-3f2d-4625-b59a-107972ae91b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:04.873106 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-0002604b-3f2d-4625-b59a-107972ae91b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:04.874583 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 983/1986] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:50:04 2022] DELETE /identity/v3/projects/0d3463b978574c838892dbb1207531d1 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:50:04.879852 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3f22cc-2935-4caa-9082-44c6fbf55657 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:04.881033 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-7a3f22cc-2935-4caa-9082-44c6fbf55657 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:04.950157 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:04.966024 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3f22cc-2935-4caa-9082-44c6fbf55657 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:04.971065 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d95640-dbde-48e3-9e46-9b1f68549571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:04.971708 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d95640-dbde-48e3-9e46-9b1f68549571 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:04.971991 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d95640-dbde-48e3-9e46-9b1f68549571 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:04.972305 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d95640-dbde-48e3-9e46-9b1f68549571 admin admin] PATH_INFO: `/v3/users/7d4b373bd5c04262b5124054f1c063b8` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:04.973588 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d95640-dbde-48e3-9e46-9b1f68549571 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d4b373bd5c04262b5124054f1c063b8)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:04.975582 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d95640-dbde-48e3-9e46-9b1f68549571 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:05.174674 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d7d95640-dbde-48e3-9e46-9b1f68549571 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d4b373bd5c04262b5124054f1c063b8'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:05.268033 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0002604b-3f2d-4625-b59a-107972ae91b2 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:05.269619 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-0002604b-3f2d-4625-b59a-107972ae91b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:05.271660 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d7d95640-dbde-48e3-9e46-9b1f68549571 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d4b373bd5c04262b5124054f1c063b8'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:05.286578 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d7d95640-dbde-48e3-9e46-9b1f68549571 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d4b373bd5c04262b5124054f1c063b8'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:05.297671 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d7d95640-dbde-48e3-9e46-9b1f68549571 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d4b373bd5c04262b5124054f1c063b8'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:05.302790 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d7d95640-dbde-48e3-9e46-9b1f68549571 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d4b373bd5c04262b5124054f1c063b8'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:05.309133 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1004/1987] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:50:04 2022] DELETE /identity/v3/users/7d4b373bd5c04262b5124054f1c063b8 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:05.345002 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0002604b-3f2d-4625-b59a-107972ae91b2 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:05.371713 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:05.413051 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dc0c07-b464-47b3-b986-848a18f42e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:05.413837 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dc0c07-b464-47b3-b986-848a18f42e58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:05.414117 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dc0c07-b464-47b3-b986-848a18f42e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:05.414389 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dc0c07-b464-47b3-b986-848a18f42e58 admin admin] PATH_INFO: `/v3/projects/c9acb65d67d74d5aa835a0840faf5336` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:05.415567 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dc0c07-b464-47b3-b986-848a18f42e58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9acb65d67d74d5aa835a0840faf5336)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:05.417605 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dc0c07-b464-47b3-b986-848a18f42e58 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:05.538831 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-d4dc0c07-b464-47b3-b986-848a18f42e58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9acb65d67d74d5aa835a0840faf5336', 'request_id': 'req-d4dc0c07-b464-47b3-b986-848a18f42e58'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:05.637556 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-d4dc0c07-b464-47b3-b986-848a18f42e58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9acb65d67d74d5aa835a0840faf5336', 'request_id': 'req-d4dc0c07-b464-47b3-b986-848a18f42e58'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:05.644062 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-d4dc0c07-b464-47b3-b986-848a18f42e58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9acb65d67d74d5aa835a0840faf5336', 'request_id': 'req-d4dc0c07-b464-47b3-b986-848a18f42e58'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:05.657101 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-d4dc0c07-b464-47b3-b986-848a18f42e58 admin admin] The token cache is being invalidate because project c9acb65d67d74d5aa835a0840faf5336 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:50:05.657387 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-d4dc0c07-b464-47b3-b986-848a18f42e58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:05.658026 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-d4dc0c07-b464-47b3-b986-848a18f42e58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:05.659544 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 984/1988] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:50:05 2022] DELETE /identity/v3/projects/c9acb65d67d74d5aa835a0840faf5336 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:50:05.664584 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd99ca-3ab5-4b6c-a8e8-39667a7aeb0a None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:05.665080 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd99ca-3ab5-4b6c-a8e8-39667a7aeb0a None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:05.665169 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd99ca-3ab5-4b6c-a8e8-39667a7aeb0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:05.685248 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-accd99ca-3ab5-4b6c-a8e8-39667a7aeb0a None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:05.686472 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-accd99ca-3ab5-4b6c-a8e8-39667a7aeb0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:05.747310 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1005/1989] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:50:05.756967 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd53bbb-f499-41ff-8785-a119bc34b275 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:05.756967 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd53bbb-f499-41ff-8785-a119bc34b275 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:05.756967 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd53bbb-f499-41ff-8785-a119bc34b275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:05.764088 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accd99ca-3ab5-4b6c-a8e8-39667a7aeb0a None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:05.766564 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-accd99ca-3ab5-4b6c-a8e8-39667a7aeb0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:05.784242 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accd99ca-3ab5-4b6c-a8e8-39667a7aeb0a None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:05.786519 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8469f2c-4b33-4bf7-a9bd-5f434e2f6f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:05.787211 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8469f2c-4b33-4bf7-a9bd-5f434e2f6f25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:05.787388 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8469f2c-4b33-4bf7-a9bd-5f434e2f6f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:05.787654 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8469f2c-4b33-4bf7-a9bd-5f434e2f6f25 admin admin] PATH_INFO: `/v3/projects/0869079882384b48b8ff7a85c1b9da8f` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:05.789050 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8469f2c-4b33-4bf7-a9bd-5f434e2f6f25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0869079882384b48b8ff7a85c1b9da8f)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:05.790706 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-dbd53bbb-f499-41ff-8785-a119bc34b275 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:05.791164 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8469f2c-4b33-4bf7-a9bd-5f434e2f6f25 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:05.791922 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-dbd53bbb-f499-41ff-8785-a119bc34b275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:05.801122 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 985/1990] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:50:05.805300 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd53bbb-f499-41ff-8785-a119bc34b275 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:05.814418 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd53bbb-f499-41ff-8785-a119bc34b275 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:05.819838 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e32e7-3f89-48ef-aec4-948c205c5962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:05.820441 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e32e7-3f89-48ef-aec4-948c205c5962 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:05.820746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e32e7-3f89-48ef-aec4-948c205c5962 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:05.821062 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e32e7-3f89-48ef-aec4-948c205c5962 admin admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:05.821715 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436e32e7-3f89-48ef-aec4-948c205c5962 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:50:05.822022 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436e32e7-3f89-48ef-aec4-948c205c5962 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:05.823626 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436e32e7-3f89-48ef-aec4-948c205c5962 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:05.836444 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 986/1991] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:50:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:50:05.841689 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e32e7-3f89-48ef-aec4-948c205c5962 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:05.850612 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e32e7-3f89-48ef-aec4-948c205c5962 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:05.855288 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5d8b3-b64e-4a28-942f-5727ea4fb6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:05.856002 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5d8b3-b64e-4a28-942f-5727ea4fb6f6 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:05.856284 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5d8b3-b64e-4a28-942f-5727ea4fb6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:05.856580 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5d8b3-b64e-4a28-942f-5727ea4fb6f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:05.857233 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff5d8b3-b64e-4a28-942f-5727ea4fb6f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:05.859120 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff5d8b3-b64e-4a28-942f-5727ea4fb6f6 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:05.901301 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-a8469f2c-4b33-4bf7-a9bd-5f434e2f6f25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0869079882384b48b8ff7a85c1b9da8f', 'request_id': 'req-a8469f2c-4b33-4bf7-a9bd-5f434e2f6f25'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:05.906713 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-a8469f2c-4b33-4bf7-a9bd-5f434e2f6f25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0869079882384b48b8ff7a85c1b9da8f', 'request_id': 'req-a8469f2c-4b33-4bf7-a9bd-5f434e2f6f25'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:05.914220 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-a8469f2c-4b33-4bf7-a9bd-5f434e2f6f25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0869079882384b48b8ff7a85c1b9da8f', 'request_id': 'req-a8469f2c-4b33-4bf7-a9bd-5f434e2f6f25'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:05.916031 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:05.975983 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 987/1992] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:50:05 2022] POST /identity/v3/projects => generated 373 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:50:05.980902 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5d8b3-b64e-4a28-942f-5727ea4fb6f6 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:06.002833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-a8469f2c-4b33-4bf7-a9bd-5f434e2f6f25 admin admin] The token cache is being invalidate because project 0869079882384b48b8ff7a85c1b9da8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:50:06.003125 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-a8469f2c-4b33-4bf7-a9bd-5f434e2f6f25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:06.004483 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-a8469f2c-4b33-4bf7-a9bd-5f434e2f6f25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:06.006213 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1006/1993] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:50:05 2022] DELETE /identity/v3/projects/0869079882384b48b8ff7a85c1b9da8f => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:06.010882 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ca193-17dd-49a9-87c8-2964f080b046 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:06.011166 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ca193-17dd-49a9-87c8-2964f080b046 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:06.011437 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ca193-17dd-49a9-87c8-2964f080b046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:06.030779 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-874ca193-17dd-49a9-87c8-2964f080b046 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:06.031985 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-874ca193-17dd-49a9-87c8-2964f080b046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:06.046569 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5d8b3-b64e-4a28-942f-5727ea4fb6f6 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:06.052317 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42decc38-1884-4b34-903f-380e40ced684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:06.053005 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42decc38-1884-4b34-903f-380e40ced684 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:06.058297 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42decc38-1884-4b34-903f-380e40ced684 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:06.058297 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42decc38-1884-4b34-903f-380e40ced684 admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:06.058297 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42decc38-1884-4b34-903f-380e40ced684 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:06.058297 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42decc38-1884-4b34-903f-380e40ced684 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:06.095919 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1007/1994] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:50:06.114501 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf60ee2-4482-4387-8134-2c749c4fbbef None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:06.114501 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf60ee2-4482-4387-8134-2c749c4fbbef None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:06.114501 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf60ee2-4482-4387-8134-2c749c4fbbef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:06.125735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-4bf60ee2-4482-4387-8134-2c749c4fbbef None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:06.127002 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-4bf60ee2-4482-4387-8134-2c749c4fbbef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:06.135766 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1008/1995] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:50:06.140045 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf60ee2-4482-4387-8134-2c749c4fbbef None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:06.149290 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf60ee2-4482-4387-8134-2c749c4fbbef None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:06.154356 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cdf770-d106-4764-b36a-b12b10fae060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:06.154954 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cdf770-d106-4764-b36a-b12b10fae060 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:06.155251 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cdf770-d106-4764-b36a-b12b10fae060 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:06.155505 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cdf770-d106-4764-b36a-b12b10fae060 admin admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:06.156152 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cdf770-d106-4764-b36a-b12b10fae060 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:50:06.156452 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cdf770-d106-4764-b36a-b12b10fae060 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:06.158737 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 988/1996] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:50:05 2022] POST /identity/v3/users => generated 287 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:50:06.158872 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cdf770-d106-4764-b36a-b12b10fae060 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:06.164824 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42decc38-1884-4b34-903f-380e40ced684 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:06.166476 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-42decc38-1884-4b34-903f-380e40ced684 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:06.173991 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1009/1997] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:50:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:50:06.175046 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cdf770-d106-4764-b36a-b12b10fae060 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:06.179703 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42decc38-1884-4b34-903f-380e40ced684 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:06.184757 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f214d7-1c99-414f-a114-478312d115c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:06.185390 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f214d7-1c99-414f-a114-478312d115c1 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:06.185745 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f214d7-1c99-414f-a114-478312d115c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:06.185856 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cdf770-d106-4764-b36a-b12b10fae060 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:06.185939 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f214d7-1c99-414f-a114-478312d115c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:06.186653 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f214d7-1c99-414f-a114-478312d115c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:06.188241 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f214d7-1c99-414f-a114-478312d115c1 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:06.190748 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53d18a1-4d8e-405c-99ba-23c52a8c02ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:06.192222 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53d18a1-4d8e-405c-99ba-23c52a8c02ba admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:06.192507 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53d18a1-4d8e-405c-99ba-23c52a8c02ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:06.196211 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53d18a1-4d8e-405c-99ba-23c52a8c02ba admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:06.196211 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53d18a1-4d8e-405c-99ba-23c52a8c02ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:06.196211 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53d18a1-4d8e-405c-99ba-23c52a8c02ba admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:06.202936 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 989/1998] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:50:06.203991 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f214d7-1c99-414f-a114-478312d115c1 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:06.213902 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f214d7-1c99-414f-a114-478312d115c1 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:06.218482 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db7ea8-9322-45e3-8d7e-1e7908a18142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:06.219063 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db7ea8-9322-45e3-8d7e-1e7908a18142 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:06.219340 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db7ea8-9322-45e3-8d7e-1e7908a18142 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:06.219610 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db7ea8-9322-45e3-8d7e-1e7908a18142 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:06.220242 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6db7ea8-9322-45e3-8d7e-1e7908a18142 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:06.221867 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6db7ea8-9322-45e3-8d7e-1e7908a18142 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:06.232417 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 990/1999] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:50:06.236396 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db7ea8-9322-45e3-8d7e-1e7908a18142 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:06.247138 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db7ea8-9322-45e3-8d7e-1e7908a18142 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:06.251770 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832337e7-ed1b-4b37-886d-12f9694d87ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:06.252393 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832337e7-ed1b-4b37-886d-12f9694d87ba admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:06.252666 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832337e7-ed1b-4b37-886d-12f9694d87ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:06.252964 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832337e7-ed1b-4b37-886d-12f9694d87ba admin admin] PATH_INFO: `/v3/projects/268dca13760744dfa511c456ba54fad8/users/a42beabaf88043398e48326cbde0a746/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:06.267790 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832337e7-ed1b-4b37-886d-12f9694d87ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=268dca13760744dfa511c456ba54fad8, user_id=a42beabaf88043398e48326cbde0a746, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:06.270604 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832337e7-ed1b-4b37-886d-12f9694d87ba admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:06.292678 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:06.334372 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1010/2000] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:50:06 2022] POST /identity/v3/projects => generated 383 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:50:06.338558 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53d18a1-4d8e-405c-99ba-23c52a8c02ba admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:06.367828 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 991/2001] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:50:06 2022] PUT /identity/v3/projects/268dca13760744dfa511c456ba54fad8/users/a42beabaf88043398e48326cbde0a746/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:50:06.380295 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7943a1-d31c-46f9-9675-3a18d6eba31a None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:06.380592 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7943a1-d31c-46f9-9675-3a18d6eba31a None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:06.380905 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7943a1-d31c-46f9-9675-3a18d6eba31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:06.404059 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53d18a1-4d8e-405c-99ba-23c52a8c02ba admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:06.409302 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-ee7943a1-d31c-46f9-9675-3a18d6eba31a None None] MFA Rules not processed for user `a42beabaf88043398e48326cbde0a746`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:06.411154 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-ee7943a1-d31c-46f9-9675-3a18d6eba31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:06.462995 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768814a2-74a6-4e3f-a5b3-d1c86e6eedfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:06.463808 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768814a2-74a6-4e3f-a5b3-d1c86e6eedfd admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:06.464304 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768814a2-74a6-4e3f-a5b3-d1c86e6eedfd admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:06.464529 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768814a2-74a6-4e3f-a5b3-d1c86e6eedfd admin admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:06.465028 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768814a2-74a6-4e3f-a5b3-d1c86e6eedfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:06.466996 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768814a2-74a6-4e3f-a5b3-d1c86e6eedfd admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:06.493502 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 992/2002] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:50:06.505128 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7943a1-d31c-46f9-9675-3a18d6eba31a None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:06.506465 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-ee7943a1-d31c-46f9-9675-3a18d6eba31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:06.564375 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1011/2003] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:50:06 2022] POST /identity/v3/users => generated 292 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:50:06.564812 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7943a1-d31c-46f9-9675-3a18d6eba31a None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:06.568387 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768814a2-74a6-4e3f-a5b3-d1c86e6eedfd admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:06.569412 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bdf1ee-3332-45a9-a69e-f7fc9cd74c3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:06.570070 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bdf1ee-3332-45a9-a69e-f7fc9cd74c3a service cinder] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:06.570355 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bdf1ee-3332-45a9-a69e-f7fc9cd74c3a service cinder] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:06.570632 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bdf1ee-3332-45a9-a69e-f7fc9cd74c3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:06.578370 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768814a2-74a6-4e3f-a5b3-d1c86e6eedfd admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:06.579523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bdf1ee-3332-45a9-a69e-f7fc9cd74c3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:06.581320 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bdf1ee-3332-45a9-a69e-f7fc9cd74c3a service cinder] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:06.583708 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f5e1a4-0591-4194-8d60-43fc1760d8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:06.584454 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5e1a4-0591-4194-8d60-43fc1760d8f6 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:06.584804 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5e1a4-0591-4194-8d60-43fc1760d8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:06.585092 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5e1a4-0591-4194-8d60-43fc1760d8f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:06.585850 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f5e1a4-0591-4194-8d60-43fc1760d8f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:06.587430 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f5e1a4-0591-4194-8d60-43fc1760d8f6 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:06.591088 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 993/2004] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:50:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:50:06.604142 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1012/2005] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:50:06.605363 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bdf1ee-3332-45a9-a69e-f7fc9cd74c3a service cinder] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:06.614688 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f5e1a4-0591-4194-8d60-43fc1760d8f6 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:06.621032 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bdf1ee-3332-45a9-a69e-f7fc9cd74c3a service cinder] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:06.624823 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f5e1a4-0591-4194-8d60-43fc1760d8f6 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:06.625786 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60e52e-d3ea-40f2-894c-c731896f6419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:06.626390 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60e52e-d3ea-40f2-894c-c731896f6419 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:06.626671 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60e52e-d3ea-40f2-894c-c731896f6419 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:06.626947 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60e52e-d3ea-40f2-894c-c731896f6419 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:06.627587 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c60e52e-d3ea-40f2-894c-c731896f6419 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:06.629813 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e811393-1f1f-4144-8d45-2422a7cb5f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:06.629949 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c60e52e-d3ea-40f2-894c-c731896f6419 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:06.630555 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e811393-1f1f-4144-8d45-2422a7cb5f57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:06.630899 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e811393-1f1f-4144-8d45-2422a7cb5f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:06.631252 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e811393-1f1f-4144-8d45-2422a7cb5f57 admin admin] PATH_INFO: `/v3/users/a42beabaf88043398e48326cbde0a746` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:06.632440 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e811393-1f1f-4144-8d45-2422a7cb5f57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a42beabaf88043398e48326cbde0a746)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:06.634526 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e811393-1f1f-4144-8d45-2422a7cb5f57 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:06.640440 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 994/2006] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:50:06.645339 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60e52e-d3ea-40f2-894c-c731896f6419 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:06.655150 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60e52e-d3ea-40f2-894c-c731896f6419 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:06.660837 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a101eb-b60b-4872-800f-7fbbef225a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:06.661694 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a101eb-b60b-4872-800f-7fbbef225a1b admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:06.661977 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a101eb-b60b-4872-800f-7fbbef225a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:06.662251 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a101eb-b60b-4872-800f-7fbbef225a1b admin admin] PATH_INFO: `/v3/projects/03cc5152d3194d9fb19866b943beb8d4/users/5053d65bfcfa4ee7b8ce32160ddf98f8/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:06.677782 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a101eb-b60b-4872-800f-7fbbef225a1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=03cc5152d3194d9fb19866b943beb8d4, user_id=5053d65bfcfa4ee7b8ce32160ddf98f8, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:06.681671 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a101eb-b60b-4872-800f-7fbbef225a1b admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:06.775262 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 995/2007] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:50:06 2022] PUT /identity/v3/projects/03cc5152d3194d9fb19866b943beb8d4/users/5053d65bfcfa4ee7b8ce32160ddf98f8/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:50:06.787631 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0fa50-cab5-4e1b-99ce-614acc40b0be None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:06.787916 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0fa50-cab5-4e1b-99ce-614acc40b0be None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:06.788191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0fa50-cab5-4e1b-99ce-614acc40b0be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:06.817313 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-90f0fa50-cab5-4e1b-99ce-614acc40b0be None None] MFA Rules not processed for user `5053d65bfcfa4ee7b8ce32160ddf98f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:06.818528 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-90f0fa50-cab5-4e1b-99ce-614acc40b0be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:06.829169 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-3e811393-1f1f-4144-8d45-2422a7cb5f57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a42beabaf88043398e48326cbde0a746'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:06.882853 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 996/2008] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:50:06.887524 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f0fa50-cab5-4e1b-99ce-614acc40b0be None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:06.917233 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-3e811393-1f1f-4144-8d45-2422a7cb5f57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a42beabaf88043398e48326cbde0a746'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:06.930891 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-3e811393-1f1f-4144-8d45-2422a7cb5f57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a42beabaf88043398e48326cbde0a746'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:06.938213 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-3e811393-1f1f-4144-8d45-2422a7cb5f57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a42beabaf88043398e48326cbde0a746'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:06.939069 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-3e811393-1f1f-4144-8d45-2422a7cb5f57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a42beabaf88043398e48326cbde0a746'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:06.950478 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1013/2009] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:50:06 2022] DELETE /identity/v3/users/a42beabaf88043398e48326cbde0a746 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:06.951289 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f0fa50-cab5-4e1b-99ce-614acc40b0be None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:06.953341 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081063b1-50f6-4564-a6f3-823cc0814f52 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:06.953341 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081063b1-50f6-4564-a6f3-823cc0814f52 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:06.953674 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081063b1-50f6-4564-a6f3-823cc0814f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:06.971959 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:INSERT=2,keystone:DELETE=1,keystone:SELECT=97 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:06.979286 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-081063b1-50f6-4564-a6f3-823cc0814f52 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:06.981606 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-081063b1-50f6-4564-a6f3-823cc0814f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:07.013595 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:07.017810 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36e63b9-a1ae-4d77-be1c-1d1dd3895a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:07.018461 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36e63b9-a1ae-4d77-be1c-1d1dd3895a38 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:07.018729 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36e63b9-a1ae-4d77-be1c-1d1dd3895a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:07.018996 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36e63b9-a1ae-4d77-be1c-1d1dd3895a38 admin admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:07.020342 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36e63b9-a1ae-4d77-be1c-1d1dd3895a38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:07.022421 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36e63b9-a1ae-4d77-be1c-1d1dd3895a38 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:07.063212 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1014/2010] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:50:07.073613 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081063b1-50f6-4564-a6f3-823cc0814f52 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:07.074953 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-081063b1-50f6-4564-a6f3-823cc0814f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:07.137005 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081063b1-50f6-4564-a6f3-823cc0814f52 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:07.138096 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc32ffa0-6dc3-4d99-a1fc-0b5170209a8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:07.138778 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc32ffa0-6dc3-4d99-a1fc-0b5170209a8d service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:07.139120 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc32ffa0-6dc3-4d99-a1fc-0b5170209a8d service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:07.139404 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc32ffa0-6dc3-4d99-a1fc-0b5170209a8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:07.139802 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 997/2011] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:50:06 2022] POST /identity/v3/projects => generated 379 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:50:07.143888 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36e63b9-a1ae-4d77-be1c-1d1dd3895a38 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:07.146432 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-d36e63b9-a1ae-4d77-be1c-1d1dd3895a38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:07.151279 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc32ffa0-6dc3-4d99-a1fc-0b5170209a8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:07.152709 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc32ffa0-6dc3-4d99-a1fc-0b5170209a8d service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:07.163468 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1015/2012] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:50:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:50:07.219217 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36e63b9-a1ae-4d77-be1c-1d1dd3895a38 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:07.225701 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d374ab0-0abb-4111-aec8-d3a32fca8fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:07.231917 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d374ab0-0abb-4111-aec8-d3a32fca8fbd admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:07.232119 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d374ab0-0abb-4111-aec8-d3a32fca8fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:07.232341 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d374ab0-0abb-4111-aec8-d3a32fca8fbd admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:07.233177 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d374ab0-0abb-4111-aec8-d3a32fca8fbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:07.235173 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d374ab0-0abb-4111-aec8-d3a32fca8fbd admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:07.301247 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:07.335653 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 998/2013] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:50:07 2022] POST /identity/v3/users => generated 296 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:50:07.339548 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc32ffa0-6dc3-4d99-a1fc-0b5170209a8d service neutron] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:07.350197 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc32ffa0-6dc3-4d99-a1fc-0b5170209a8d service neutron] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:07.355138 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af24bba3-dee7-416a-b503-920132c931c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:07.355956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24bba3-dee7-416a-b503-920132c931c1 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:07.356595 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24bba3-dee7-416a-b503-920132c931c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:07.358302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:07.360045 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24bba3-dee7-416a-b503-920132c931c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:07.361045 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af24bba3-dee7-416a-b503-920132c931c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:07.362895 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af24bba3-dee7-416a-b503-920132c931c1 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:07.379030 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1016/2014] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:07 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:50:07.380503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d374ab0-0abb-4111-aec8-d3a32fca8fbd admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:07.391511 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d374ab0-0abb-4111-aec8-d3a32fca8fbd admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:07.396659 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfde9cf-5f53-46a1-b04f-a561d692ea2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:07.397363 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfde9cf-5f53-46a1-b04f-a561d692ea2b admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:07.397640 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfde9cf-5f53-46a1-b04f-a561d692ea2b admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:07.397912 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfde9cf-5f53-46a1-b04f-a561d692ea2b admin admin] PATH_INFO: `/v3/projects/8674202148024dd19bf52004e35e6764/users/354734f9073c4db88a83f0b449178747/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:07.416099 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfde9cf-5f53-46a1-b04f-a561d692ea2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8674202148024dd19bf52004e35e6764, user_id=354734f9073c4db88a83f0b449178747, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:07.418791 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfde9cf-5f53-46a1-b04f-a561d692ea2b admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:07.509691 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 999/2015] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:50:07 2022] PUT /identity/v3/projects/8674202148024dd19bf52004e35e6764/users/354734f9073c4db88a83f0b449178747/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:07.512298 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af24bba3-dee7-416a-b503-920132c931c1 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:07.524559 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af24bba3-dee7-416a-b503-920132c931c1 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:07.584037 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b03fb-5e56-48a0-9495-79f4b922e852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:07.584854 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b03fb-5e56-48a0-9495-79f4b922e852 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:07.585038 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b03fb-5e56-48a0-9495-79f4b922e852 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:07.595410 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b03fb-5e56-48a0-9495-79f4b922e852 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:07.595410 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b03fb-5e56-48a0-9495-79f4b922e852 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:07.595410 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b03fb-5e56-48a0-9495-79f4b922e852 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:07.598177 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1017/2016] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:07 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:50:07.602648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfde9cf-5f53-46a1-b04f-a561d692ea2b admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:07.612866 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfde9cf-5f53-46a1-b04f-a561d692ea2b admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:07.618691 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b5807d-8569-4d31-8aa5-0227d5794775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:07.619411 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b5807d-8569-4d31-8aa5-0227d5794775 admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:07.619702 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b5807d-8569-4d31-8aa5-0227d5794775 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:07.619979 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b5807d-8569-4d31-8aa5-0227d5794775 admin admin] PATH_INFO: `/v3/projects/8674202148024dd19bf52004e35e6764/users/354734f9073c4db88a83f0b449178747/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:07.625306 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b5807d-8569-4d31-8aa5-0227d5794775 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8674202148024dd19bf52004e35e6764, user_id=354734f9073c4db88a83f0b449178747, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:07.625306 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b5807d-8569-4d31-8aa5-0227d5794775 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:07.637947 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1000/2017] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:50:07 2022] PUT /identity/v3/projects/8674202148024dd19bf52004e35e6764/users/354734f9073c4db88a83f0b449178747/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:50:07.653438 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c5b02-eadf-474a-b457-6e99b1522768 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:07.653438 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c5b02-eadf-474a-b457-6e99b1522768 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:07.653438 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c5b02-eadf-474a-b457-6e99b1522768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:07.656317 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b5807d-8569-4d31-8aa5-0227d5794775 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:07.662469 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-98b5807d-8569-4d31-8aa5-0227d5794775 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:07.693845 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-bc4c5b02-eadf-474a-b457-6e99b1522768 None None] MFA Rules not processed for user `354734f9073c4db88a83f0b449178747`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:07.695364 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-bc4c5b02-eadf-474a-b457-6e99b1522768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:07.756661 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b5807d-8569-4d31-8aa5-0227d5794775 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:07.761590 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0847d0e2-6212-4f0d-bec8-ac22b82940d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:07.762230 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847d0e2-6212-4f0d-bec8-ac22b82940d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:07.762520 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847d0e2-6212-4f0d-bec8-ac22b82940d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:07.762785 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847d0e2-6212-4f0d-bec8-ac22b82940d0 admin admin] PATH_INFO: `/v3/projects/268dca13760744dfa511c456ba54fad8` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:07.763958 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0847d0e2-6212-4f0d-bec8-ac22b82940d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=268dca13760744dfa511c456ba54fad8)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:07.766084 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0847d0e2-6212-4f0d-bec8-ac22b82940d0 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:07.777172 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1018/2018] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:07 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:50:07.791257 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c5b02-eadf-474a-b457-6e99b1522768 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:07.792669 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-bc4c5b02-eadf-474a-b457-6e99b1522768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:07.855056 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c5b02-eadf-474a-b457-6e99b1522768 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:07.858645 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1fcc42-3828-48d3-882a-a5a71a135406 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:07.859273 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1fcc42-3828-48d3-882a-a5a71a135406 service cinder] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:07.859552 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1fcc42-3828-48d3-882a-a5a71a135406 service cinder] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:07.859826 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1fcc42-3828-48d3-882a-a5a71a135406 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:07.870353 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1fcc42-3828-48d3-882a-a5a71a135406 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:07.871780 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1fcc42-3828-48d3-882a-a5a71a135406 service cinder] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:07.882393 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1019/2019] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:50:07 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:50:07.891064 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-0847d0e2-6212-4f0d-bec8-ac22b82940d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '268dca13760744dfa511c456ba54fad8', 'request_id': 'req-0847d0e2-6212-4f0d-bec8-ac22b82940d0'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:07.980562 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-0847d0e2-6212-4f0d-bec8-ac22b82940d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '268dca13760744dfa511c456ba54fad8', 'request_id': 'req-0847d0e2-6212-4f0d-bec8-ac22b82940d0'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:07.987221 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-0847d0e2-6212-4f0d-bec8-ac22b82940d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '268dca13760744dfa511c456ba54fad8', 'request_id': 'req-0847d0e2-6212-4f0d-bec8-ac22b82940d0'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:08.000796 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-0847d0e2-6212-4f0d-bec8-ac22b82940d0 admin admin] The token cache is being invalidate because project 268dca13760744dfa511c456ba54fad8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:50:08.001115 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-0847d0e2-6212-4f0d-bec8-ac22b82940d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:08.001887 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-0847d0e2-6212-4f0d-bec8-ac22b82940d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:08.003272 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1001/2020] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:50:07 2022] DELETE /identity/v3/projects/268dca13760744dfa511c456ba54fad8 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:08.008857 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b3f99-ed8e-4549-9246-b3f94d911284 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:08.009142 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b3f99-ed8e-4549-9246-b3f94d911284 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:08.009488 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b3f99-ed8e-4549-9246-b3f94d911284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:08.018121 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:08.029530 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-e46b3f99-ed8e-4549-9246-b3f94d911284 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:08.030762 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-e46b3f99-ed8e-4549-9246-b3f94d911284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:08.070181 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0847d0e2-6212-4f0d-bec8-ac22b82940d0 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:08.071654 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-0847d0e2-6212-4f0d-bec8-ac22b82940d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:08.091613 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1020/2021] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:50:08.096173 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:08.101327 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05410dd9-6e23-4116-a022-a5737506cdcf None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:08.101614 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05410dd9-6e23-4116-a022-a5737506cdcf None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:08.101890 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05410dd9-6e23-4116-a022-a5737506cdcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:08.122151 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-05410dd9-6e23-4116-a022-a5737506cdcf None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:08.123346 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-05410dd9-6e23-4116-a022-a5737506cdcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:08.132588 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1021/2022] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:50:08.133944 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0847d0e2-6212-4f0d-bec8-ac22b82940d0 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:08.136642 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05410dd9-6e23-4116-a022-a5737506cdcf None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:08.138075 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2040870-37e1-41b2-a83f-a7f690d6f5a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:08.138677 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2040870-37e1-41b2-a83f-a7f690d6f5a6 service cinder] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:08.138959 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2040870-37e1-41b2-a83f-a7f690d6f5a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:08.139232 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2040870-37e1-41b2-a83f-a7f690d6f5a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:08.141023 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-e2040870-37e1-41b2-a83f-a7f690d6f5a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:08.146384 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05410dd9-6e23-4116-a022-a5737506cdcf None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:08.151612 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069294e9-f6dd-49b4-80b4-f3794f1630db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:08.152145 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069294e9-f6dd-49b4-80b4-f3794f1630db admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:08.152427 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069294e9-f6dd-49b4-80b4-f3794f1630db admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:08.152751 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069294e9-f6dd-49b4-80b4-f3794f1630db admin admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:08.153406 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069294e9-f6dd-49b4-80b4-f3794f1630db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:50:08.153715 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069294e9-f6dd-49b4-80b4-f3794f1630db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:08.155390 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069294e9-f6dd-49b4-80b4-f3794f1630db admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:08.171057 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1022/2023] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:50:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:50:08.172262 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069294e9-f6dd-49b4-80b4-f3794f1630db admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:08.183258 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069294e9-f6dd-49b4-80b4-f3794f1630db admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:08.189199 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b69296-6c7d-4089-80c9-f6fb339a18a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:08.189369 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b69296-6c7d-4089-80c9-f6fb339a18a3 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:08.189648 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b69296-6c7d-4089-80c9-f6fb339a18a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:08.189923 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b69296-6c7d-4089-80c9-f6fb339a18a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:08.190594 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b69296-6c7d-4089-80c9-f6fb339a18a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:08.193224 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b69296-6c7d-4089-80c9-f6fb339a18a3 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:08.204342 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2040870-37e1-41b2-a83f-a7f690d6f5a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:08.206094 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2040870-37e1-41b2-a83f-a7f690d6f5a6 service cinder] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:08.215548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1002/2024] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:50:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:50:08.328765 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1023/2025] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:50:08 2022] POST /identity/v3/projects => generated 385 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:50:08.335864 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2040870-37e1-41b2-a83f-a7f690d6f5a6 service cinder] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:08.346812 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2040870-37e1-41b2-a83f-a7f690d6f5a6 service cinder] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:08.402408 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b766015-255e-4a2c-be7c-63512d1a1a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:08.403027 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b766015-255e-4a2c-be7c-63512d1a1a76 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:08.403308 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b766015-255e-4a2c-be7c-63512d1a1a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:08.403582 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b766015-255e-4a2c-be7c-63512d1a1a76 admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:08.404213 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b766015-255e-4a2c-be7c-63512d1a1a76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:08.406239 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b766015-255e-4a2c-be7c-63512d1a1a76 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:08.520975 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1003/2026] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:50:08 2022] POST /identity/v3/users => generated 293 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:50:08.525158 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b69296-6c7d-4089-80c9-f6fb339a18a3 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:08.535342 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b69296-6c7d-4089-80c9-f6fb339a18a3 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:08.540597 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d470996-8492-43e7-9080-fbd601ce32b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:08.541240 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d470996-8492-43e7-9080-fbd601ce32b8 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:08.541518 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d470996-8492-43e7-9080-fbd601ce32b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:08.541790 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d470996-8492-43e7-9080-fbd601ce32b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:08.542506 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d470996-8492-43e7-9080-fbd601ce32b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:08.544393 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d470996-8492-43e7-9080-fbd601ce32b8 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:08.555965 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1024/2027] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:50:08.559975 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b766015-255e-4a2c-be7c-63512d1a1a76 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:08.571994 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b766015-255e-4a2c-be7c-63512d1a1a76 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:08.577004 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408dfe50-278f-4cba-91b3-e6306bf6b476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:08.577591 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408dfe50-278f-4cba-91b3-e6306bf6b476 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:08.577866 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408dfe50-278f-4cba-91b3-e6306bf6b476 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:08.578137 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408dfe50-278f-4cba-91b3-e6306bf6b476 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:08.578878 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408dfe50-278f-4cba-91b3-e6306bf6b476 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:08.580742 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408dfe50-278f-4cba-91b3-e6306bf6b476 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:08.594943 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1004/2028] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:50:08.596216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d470996-8492-43e7-9080-fbd601ce32b8 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:08.607243 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d470996-8492-43e7-9080-fbd601ce32b8 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:08.612678 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e100ac3-5996-4479-966e-5c750465e462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:08.612678 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e100ac3-5996-4479-966e-5c750465e462 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:08.612678 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e100ac3-5996-4479-966e-5c750465e462 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:08.613796 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e100ac3-5996-4479-966e-5c750465e462 admin admin] PATH_INFO: `/v3/projects/f5d16a389a22469d983ca8fd2df60644/users/48c1580f32f049eeb37a8b5fdc38d566/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:08.629919 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e100ac3-5996-4479-966e-5c750465e462 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5d16a389a22469d983ca8fd2df60644, user_id=48c1580f32f049eeb37a8b5fdc38d566, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:08.631815 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e100ac3-5996-4479-966e-5c750465e462 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:08.787093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1025/2029] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:50:08 2022] PUT /identity/v3/projects/f5d16a389a22469d983ca8fd2df60644/users/48c1580f32f049eeb37a8b5fdc38d566/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:08.797925 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454293a-dd5e-4cce-b2f0-48c8ff559e9b None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:08.798224 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454293a-dd5e-4cce-b2f0-48c8ff559e9b None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:08.798499 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454293a-dd5e-4cce-b2f0-48c8ff559e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:08.831088 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-6454293a-dd5e-4cce-b2f0-48c8ff559e9b None None] MFA Rules not processed for user `48c1580f32f049eeb37a8b5fdc38d566`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:08.833088 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-6454293a-dd5e-4cce-b2f0-48c8ff559e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:08.850366 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e100ac3-5996-4479-966e-5c750465e462 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:08.852124 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-3e100ac3-5996-4479-966e-5c750465e462 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:08.904343 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1005/2030] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:50:08.908887 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6454293a-dd5e-4cce-b2f0-48c8ff559e9b None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:08.931103 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e100ac3-5996-4479-966e-5c750465e462 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:08.936539 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a9065-c63c-445d-ba56-0837e33ec548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:08.941538 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a9065-c63c-445d-ba56-0837e33ec548 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:08.941907 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a9065-c63c-445d-ba56-0837e33ec548 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:08.942267 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a9065-c63c-445d-ba56-0837e33ec548 admin admin] PATH_INFO: `/v3/users/5053d65bfcfa4ee7b8ce32160ddf98f8` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:08.943531 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10a9065-c63c-445d-ba56-0837e33ec548 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5053d65bfcfa4ee7b8ce32160ddf98f8)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:08.945765 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10a9065-c63c-445d-ba56-0837e33ec548 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:08.987825 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6454293a-dd5e-4cce-b2f0-48c8ff559e9b None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:08.992851 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5216beb-d0ca-4ed9-81af-c2d0b5447153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:08.993496 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5216beb-d0ca-4ed9-81af-c2d0b5447153 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:08.993890 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5216beb-d0ca-4ed9-81af-c2d0b5447153 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:08.994046 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5216beb-d0ca-4ed9-81af-c2d0b5447153 admin admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:08.994671 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5216beb-d0ca-4ed9-81af-c2d0b5447153 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:08.996576 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5216beb-d0ca-4ed9-81af-c2d0b5447153 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:09.148114 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1006/2031] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:50:08 2022] POST /identity/v3/projects => generated 383 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:50:09.152099 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5216beb-d0ca-4ed9-81af-c2d0b5447153 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:09.181360 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:09.183555 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:09.194964 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-f10a9065-c63c-445d-ba56-0837e33ec548 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5053d65bfcfa4ee7b8ce32160ddf98f8'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:09.219484 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5216beb-d0ca-4ed9-81af-c2d0b5447153 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:09.225863 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136f1103-ee4f-4b3f-8dda-cc51cea3d191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:09.226481 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136f1103-ee4f-4b3f-8dda-cc51cea3d191 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:09.226795 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136f1103-ee4f-4b3f-8dda-cc51cea3d191 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:09.229113 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136f1103-ee4f-4b3f-8dda-cc51cea3d191 admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:09.229113 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136f1103-ee4f-4b3f-8dda-cc51cea3d191 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:09.229757 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136f1103-ee4f-4b3f-8dda-cc51cea3d191 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:09.277464 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-f10a9065-c63c-445d-ba56-0837e33ec548 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5053d65bfcfa4ee7b8ce32160ddf98f8'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:09.291100 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-f10a9065-c63c-445d-ba56-0837e33ec548 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5053d65bfcfa4ee7b8ce32160ddf98f8'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:09.298110 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-f10a9065-c63c-445d-ba56-0837e33ec548 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5053d65bfcfa4ee7b8ce32160ddf98f8'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:09.298885 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-f10a9065-c63c-445d-ba56-0837e33ec548 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5053d65bfcfa4ee7b8ce32160ddf98f8'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:09.305326 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1026/2032] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:50:08 2022] DELETE /identity/v3/users/5053d65bfcfa4ee7b8ce32160ddf98f8 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:09.309898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287049f2-c690-4a83-a0dd-33be401ba353 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:09.310188 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287049f2-c690-4a83-a0dd-33be401ba353 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:09.310535 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287049f2-c690-4a83-a0dd-33be401ba353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:09.319764 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1007/2033] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:50:09 2022] POST /identity/v3/users => generated 292 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:50:09.324091 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136f1103-ee4f-4b3f-8dda-cc51cea3d191 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:09.325845 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-136f1103-ee4f-4b3f-8dda-cc51cea3d191 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:09.332509 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-287049f2-c690-4a83-a0dd-33be401ba353 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:09.334010 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-287049f2-c690-4a83-a0dd-33be401ba353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:09.342636 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136f1103-ee4f-4b3f-8dda-cc51cea3d191 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:09.402933 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedcb558-f7ac-4b28-8844-64cf0cb586f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:09.403708 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedcb558-f7ac-4b28-8844-64cf0cb586f8 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:09.403992 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedcb558-f7ac-4b28-8844-64cf0cb586f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:09.404265 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedcb558-f7ac-4b28-8844-64cf0cb586f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:09.404941 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedcb558-f7ac-4b28-8844-64cf0cb586f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:09.406572 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedcb558-f7ac-4b28-8844-64cf0cb586f8 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:09.408370 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1027/2034] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:50:09.419232 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1008/2035] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:50:09.422455 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287049f2-c690-4a83-a0dd-33be401ba353 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:09.433338 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287049f2-c690-4a83-a0dd-33be401ba353 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:09.435473 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedcb558-f7ac-4b28-8844-64cf0cb586f8 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:09.437068 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-bedcb558-f7ac-4b28-8844-64cf0cb586f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:09.438856 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f1abb3-382f-456d-b3d4-8ea76ee2c305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:09.439566 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1abb3-382f-456d-b3d4-8ea76ee2c305 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:09.439972 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1abb3-382f-456d-b3d4-8ea76ee2c305 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:09.440288 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1abb3-382f-456d-b3d4-8ea76ee2c305 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:09.441257 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f1abb3-382f-456d-b3d4-8ea76ee2c305 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:09.443155 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f1abb3-382f-456d-b3d4-8ea76ee2c305 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:09.453767 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1028/2036] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:50:09.458150 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f1abb3-382f-456d-b3d4-8ea76ee2c305 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:09.468629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f1abb3-382f-456d-b3d4-8ea76ee2c305 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:09.475276 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23268ffb-7e01-4574-ac81-3e509993a12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:09.475942 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23268ffb-7e01-4574-ac81-3e509993a12e admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:09.476225 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23268ffb-7e01-4574-ac81-3e509993a12e admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:09.476502 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23268ffb-7e01-4574-ac81-3e509993a12e admin admin] PATH_INFO: `/v3/projects/5c289e6fa3ff40feaf37d07155e7da23/users/df4177fc0f844dfd9d32bbaf175b4004/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:09.498983 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23268ffb-7e01-4574-ac81-3e509993a12e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c289e6fa3ff40feaf37d07155e7da23, user_id=df4177fc0f844dfd9d32bbaf175b4004, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:09.501753 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23268ffb-7e01-4574-ac81-3e509993a12e admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:09.504225 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedcb558-f7ac-4b28-8844-64cf0cb586f8 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:09.508293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9398b8-0d6b-46fd-8749-844e59100bb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:09.508964 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9398b8-0d6b-46fd-8749-844e59100bb6 service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:09.509247 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9398b8-0d6b-46fd-8749-844e59100bb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:09.509520 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9398b8-0d6b-46fd-8749-844e59100bb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:09.520751 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9398b8-0d6b-46fd-8749-844e59100bb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:09.522339 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9398b8-0d6b-46fd-8749-844e59100bb6 service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:09.532978 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1009/2037] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:50:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:50:09.664973 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1029/2038] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:50:09 2022] PUT /identity/v3/projects/5c289e6fa3ff40feaf37d07155e7da23/users/df4177fc0f844dfd9d32bbaf175b4004/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:50:09.679548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4215a4-08ae-441b-ba72-9ec72a34d9fc None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:09.679838 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4215a4-08ae-441b-ba72-9ec72a34d9fc None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:09.680116 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4215a4-08ae-441b-ba72-9ec72a34d9fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:09.699513 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:09.713667 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-6f4215a4-08ae-441b-ba72-9ec72a34d9fc None None] MFA Rules not processed for user `df4177fc0f844dfd9d32bbaf175b4004`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:09.713667 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-6f4215a4-08ae-441b-ba72-9ec72a34d9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:09.783872 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1010/2039] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:50:09.788572 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23268ffb-7e01-4574-ac81-3e509993a12e admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:09.857299 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23268ffb-7e01-4574-ac81-3e509993a12e admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:09.862208 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c7fa7-6a6d-4317-9731-e1a9e351d588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:09.863378 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c7fa7-6a6d-4317-9731-e1a9e351d588 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:09.863814 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c7fa7-6a6d-4317-9731-e1a9e351d588 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:09.864112 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c7fa7-6a6d-4317-9731-e1a9e351d588 admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:09.864789 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44c7fa7-6a6d-4317-9731-e1a9e351d588 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:09.866701 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44c7fa7-6a6d-4317-9731-e1a9e351d588 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:09.984318 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:09.995856 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1030/2040] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:50:09 2022] POST /identity/v3/projects => generated 385 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:50:09.997435 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4215a4-08ae-441b-ba72-9ec72a34d9fc None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:10.012363 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4215a4-08ae-441b-ba72-9ec72a34d9fc None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:10.085319 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0900be-3b8f-4a00-a4db-86ccb4066ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:10.085319 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0900be-3b8f-4a00-a4db-86ccb4066ea8 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:10.085319 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0900be-3b8f-4a00-a4db-86ccb4066ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:10.085594 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0900be-3b8f-4a00-a4db-86ccb4066ea8 admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:10.086056 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0900be-3b8f-4a00-a4db-86ccb4066ea8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:10.087999 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0900be-3b8f-4a00-a4db-86ccb4066ea8 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:10.117129 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c7fa7-6a6d-4317-9731-e1a9e351d588 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:10.124028 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-e44c7fa7-6a6d-4317-9731-e1a9e351d588 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:10.147133 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c7fa7-6a6d-4317-9731-e1a9e351d588 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:10.153527 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b301c991-e3ff-49b9-a64c-41eb76920714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:10.154166 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b301c991-e3ff-49b9-a64c-41eb76920714 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:10.154616 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b301c991-e3ff-49b9-a64c-41eb76920714 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:10.154899 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b301c991-e3ff-49b9-a64c-41eb76920714 admin admin] PATH_INFO: `/v3/projects/03cc5152d3194d9fb19866b943beb8d4` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:10.156186 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b301c991-e3ff-49b9-a64c-41eb76920714 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03cc5152d3194d9fb19866b943beb8d4)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:10.158622 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b301c991-e3ff-49b9-a64c-41eb76920714 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:10.205664 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1011/2041] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:50:09 2022] POST /identity/v3/users => generated 299 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:50:10.222009 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0900be-3b8f-4a00-a4db-86ccb4066ea8 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:10.233107 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0900be-3b8f-4a00-a4db-86ccb4066ea8 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:10.237677 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fa562a-967f-4f7a-8c5b-c03bb5aa7be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:10.239148 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa562a-967f-4f7a-8c5b-c03bb5aa7be9 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:10.241758 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa562a-967f-4f7a-8c5b-c03bb5aa7be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:10.241758 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa562a-967f-4f7a-8c5b-c03bb5aa7be9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:10.241758 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fa562a-967f-4f7a-8c5b-c03bb5aa7be9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:10.242483 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fa562a-967f-4f7a-8c5b-c03bb5aa7be9 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:10.255658 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1012/2042] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:50:10.256683 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fa562a-967f-4f7a-8c5b-c03bb5aa7be9 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:10.263569 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:10.267384 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fa562a-967f-4f7a-8c5b-c03bb5aa7be9 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:10.286928 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-b301c991-e3ff-49b9-a64c-41eb76920714 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03cc5152d3194d9fb19866b943beb8d4', 'request_id': 'req-b301c991-e3ff-49b9-a64c-41eb76920714'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:10.296731 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-b301c991-e3ff-49b9-a64c-41eb76920714 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03cc5152d3194d9fb19866b943beb8d4', 'request_id': 'req-b301c991-e3ff-49b9-a64c-41eb76920714'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:10.311586 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-b301c991-e3ff-49b9-a64c-41eb76920714 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03cc5152d3194d9fb19866b943beb8d4', 'request_id': 'req-b301c991-e3ff-49b9-a64c-41eb76920714'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:10.336442 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4c20a9-1563-400c-8f41-9d91755a7f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:10.337125 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c20a9-1563-400c-8f41-9d91755a7f45 admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:10.337447 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c20a9-1563-400c-8f41-9d91755a7f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:10.337711 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c20a9-1563-400c-8f41-9d91755a7f45 admin admin] PATH_INFO: `/v3/projects/0c83a8bd08254a3181e3a6f3768b81f8/users/7d02f460b0e745b09813fe405c0cb400/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:10.353463 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4c20a9-1563-400c-8f41-9d91755a7f45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c83a8bd08254a3181e3a6f3768b81f8, user_id=7d02f460b0e745b09813fe405c0cb400, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:10.356459 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4c20a9-1563-400c-8f41-9d91755a7f45 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:10.395599 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-b301c991-e3ff-49b9-a64c-41eb76920714 admin admin] The token cache is being invalidate because project 03cc5152d3194d9fb19866b943beb8d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:50:10.395931 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-b301c991-e3ff-49b9-a64c-41eb76920714 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:10.396652 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-b301c991-e3ff-49b9-a64c-41eb76920714 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:10.398242 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1031/2043] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:50:10 2022] DELETE /identity/v3/projects/03cc5152d3194d9fb19866b943beb8d4 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:10.402293 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b301c991-e3ff-49b9-a64c-41eb76920714 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:10.403664 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-b301c991-e3ff-49b9-a64c-41eb76920714 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:10.416591 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b301c991-e3ff-49b9-a64c-41eb76920714 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:10.421610 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7800ec88-c8b4-4c94-a6e9-012c135ec40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:10.422642 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800ec88-c8b4-4c94-a6e9-012c135ec40b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:10.422642 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800ec88-c8b4-4c94-a6e9-012c135ec40b admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:10.422882 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800ec88-c8b4-4c94-a6e9-012c135ec40b admin admin] PATH_INFO: `/v3/users/354734f9073c4db88a83f0b449178747` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:10.424055 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7800ec88-c8b4-4c94-a6e9-012c135ec40b admin admin] RBAC: Authorizing `identity:delete_user(user_id=354734f9073c4db88a83f0b449178747)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:10.426577 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7800ec88-c8b4-4c94-a6e9-012c135ec40b admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:10.428512 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1013/2044] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:50:10 2022] PUT /identity/v3/projects/0c83a8bd08254a3181e3a6f3768b81f8/users/7d02f460b0e745b09813fe405c0cb400/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:50:10.432610 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4c20a9-1563-400c-8f41-9d91755a7f45 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:10.435805 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-ce4c20a9-1563-400c-8f41-9d91755a7f45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:10.503055 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4c20a9-1563-400c-8f41-9d91755a7f45 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:10.509227 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466cfe74-186f-4417-93c6-bdc3236ea5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:10.509962 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466cfe74-186f-4417-93c6-bdc3236ea5d7 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:10.510357 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466cfe74-186f-4417-93c6-bdc3236ea5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:10.510705 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466cfe74-186f-4417-93c6-bdc3236ea5d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:10.511503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466cfe74-186f-4417-93c6-bdc3236ea5d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:10.513250 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466cfe74-186f-4417-93c6-bdc3236ea5d7 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:10.525118 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1014/2045] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:10 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:50:10.526091 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466cfe74-186f-4417-93c6-bdc3236ea5d7 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:10.527905 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-466cfe74-186f-4417-93c6-bdc3236ea5d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:10.546060 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466cfe74-186f-4417-93c6-bdc3236ea5d7 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:10.552231 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d413d7-2569-4c71-868e-fa98975ae505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:10.553062 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d413d7-2569-4c71-868e-fa98975ae505 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:10.553348 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d413d7-2569-4c71-868e-fa98975ae505 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:10.553678 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d413d7-2569-4c71-868e-fa98975ae505 admin admin] PATH_INFO: `/v3/users/82831a7fb04e4a94b767e957b4574897` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:10.554883 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d413d7-2569-4c71-868e-fa98975ae505 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82831a7fb04e4a94b767e957b4574897)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:10.556841 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d413d7-2569-4c71-868e-fa98975ae505 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:10.672202 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-7800ec88-c8b4-4c94-a6e9-012c135ec40b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '354734f9073c4db88a83f0b449178747'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:10.743632 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-7800ec88-c8b4-4c94-a6e9-012c135ec40b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '354734f9073c4db88a83f0b449178747'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:10.756899 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-7800ec88-c8b4-4c94-a6e9-012c135ec40b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '354734f9073c4db88a83f0b449178747'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:10.763925 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-7800ec88-c8b4-4c94-a6e9-012c135ec40b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '354734f9073c4db88a83f0b449178747'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:10.764672 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-7800ec88-c8b4-4c94-a6e9-012c135ec40b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '354734f9073c4db88a83f0b449178747'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:10.778404 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1032/2046] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:50:10 2022] DELETE /identity/v3/users/354734f9073c4db88a83f0b449178747 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:10.782566 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7800ec88-c8b4-4c94-a6e9-012c135ec40b admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:10.784290 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-7800ec88-c8b4-4c94-a6e9-012c135ec40b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:10.809819 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-74d413d7-2569-4c71-868e-fa98975ae505 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82831a7fb04e4a94b767e957b4574897'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:10.826006 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-74d413d7-2569-4c71-868e-fa98975ae505 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82831a7fb04e4a94b767e957b4574897'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:10.835284 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-74d413d7-2569-4c71-868e-fa98975ae505 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82831a7fb04e4a94b767e957b4574897'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:10.870912 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7800ec88-c8b4-4c94-a6e9-012c135ec40b admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:10.875785 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e1ad7-f511-48a1-ac00-aa8744c58bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:10.876298 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e1ad7-f511-48a1-ac00-aa8744c58bbb admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:10.876591 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e1ad7-f511-48a1-ac00-aa8744c58bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:10.876900 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e1ad7-f511-48a1-ac00-aa8744c58bbb admin admin] PATH_INFO: `/v3/projects/0c83a8bd08254a3181e3a6f3768b81f8/users/7d02f460b0e745b09813fe405c0cb400/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:10.879765 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e1ad7-f511-48a1-ac00-aa8744c58bbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c83a8bd08254a3181e3a6f3768b81f8, user_id=7d02f460b0e745b09813fe405c0cb400, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:10.882495 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e1ad7-f511-48a1-ac00-aa8744c58bbb admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:10.897097 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1033/2047] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:50:10 2022] PUT /identity/v3/projects/0c83a8bd08254a3181e3a6f3768b81f8/users/7d02f460b0e745b09813fe405c0cb400/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:10.910262 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6b59e-d770-4db0-b47b-6d582ad4292c None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:10.910858 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6b59e-d770-4db0-b47b-6d582ad4292c None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:10.911319 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6b59e-d770-4db0-b47b-6d582ad4292c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:10.945265 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-e6e6b59e-d770-4db0-b47b-6d582ad4292c None None] MFA Rules not processed for user `7d02f460b0e745b09813fe405c0cb400`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:10.946573 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-e6e6b59e-d770-4db0-b47b-6d582ad4292c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:11.018378 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1034/2048] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:10 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:50:11.351738 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-74d413d7-2569-4c71-868e-fa98975ae505 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82831a7fb04e4a94b767e957b4574897'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:11.352350 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-74d413d7-2569-4c71-868e-fa98975ae505 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82831a7fb04e4a94b767e957b4574897'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:11.358753 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1015/2049] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:50:10 2022] DELETE /identity/v3/users/82831a7fb04e4a94b767e957b4574897 => generated 0 bytes in 833 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:11.363417 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be324255-1f58-4806-9443-50dfa37e3c22 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:11.363705 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be324255-1f58-4806-9443-50dfa37e3c22 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:11.363985 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be324255-1f58-4806-9443-50dfa37e3c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:11.388808 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-be324255-1f58-4806-9443-50dfa37e3c22 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:11.388808 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-be324255-1f58-4806-9443-50dfa37e3c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:11.426730 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d413d7-2569-4c71-868e-fa98975ae505 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:11.429617 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-74d413d7-2569-4c71-868e-fa98975ae505 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:11.468558 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1035/2050] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:50:11.481845 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be324255-1f58-4806-9443-50dfa37e3c22 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:11.485018 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-be324255-1f58-4806-9443-50dfa37e3c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:11.513422 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d413d7-2569-4c71-868e-fa98975ae505 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:11.517185 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905717a4-90d0-41c6-88ab-a54b25511da8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:11.517780 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905717a4-90d0-41c6-88ab-a54b25511da8 service cinder] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:11.518079 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905717a4-90d0-41c6-88ab-a54b25511da8 service cinder] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:11.518396 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905717a4-90d0-41c6-88ab-a54b25511da8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:11.520906 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-905717a4-90d0-41c6-88ab-a54b25511da8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:11.584401 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be324255-1f58-4806-9443-50dfa37e3c22 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:11.588549 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:11.594722 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea01fd-1001-46a3-baf7-62f1645380a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:11.595606 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea01fd-1001-46a3-baf7-62f1645380a1 service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:11.596013 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea01fd-1001-46a3-baf7-62f1645380a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:11.596377 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea01fd-1001-46a3-baf7-62f1645380a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:11.609840 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ea01fd-1001-46a3-baf7-62f1645380a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:11.611341 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ea01fd-1001-46a3-baf7-62f1645380a1 service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:11.621865 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905717a4-90d0-41c6-88ab-a54b25511da8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:11.622330 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1036/2051] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:50:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:50:11.623460 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905717a4-90d0-41c6-88ab-a54b25511da8 service cinder] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:11.641680 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1016/2052] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:50:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:50:11.675638 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:11.719338 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea01fd-1001-46a3-baf7-62f1645380a1 service neutron] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:11.719338 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-62ea01fd-1001-46a3-baf7-62f1645380a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:11.755380 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea01fd-1001-46a3-baf7-62f1645380a1 service neutron] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:11.760748 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b7f062-3a8f-49b9-b084-f29bc3b60ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:11.761401 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b7f062-3a8f-49b9-b084-f29bc3b60ff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:11.761690 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b7f062-3a8f-49b9-b084-f29bc3b60ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:11.761966 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b7f062-3a8f-49b9-b084-f29bc3b60ff6 admin admin] PATH_INFO: `/v3/projects/8674202148024dd19bf52004e35e6764` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:11.763172 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b7f062-3a8f-49b9-b084-f29bc3b60ff6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8674202148024dd19bf52004e35e6764)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:11.765406 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b7f062-3a8f-49b9-b084-f29bc3b60ff6 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:11.915004 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-03b7f062-3a8f-49b9-b084-f29bc3b60ff6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8674202148024dd19bf52004e35e6764', 'request_id': 'req-03b7f062-3a8f-49b9-b084-f29bc3b60ff6'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:11.922952 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-03b7f062-3a8f-49b9-b084-f29bc3b60ff6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8674202148024dd19bf52004e35e6764', 'request_id': 'req-03b7f062-3a8f-49b9-b084-f29bc3b60ff6'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:11.931774 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-03b7f062-3a8f-49b9-b084-f29bc3b60ff6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8674202148024dd19bf52004e35e6764', 'request_id': 'req-03b7f062-3a8f-49b9-b084-f29bc3b60ff6'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:12.018711 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-03b7f062-3a8f-49b9-b084-f29bc3b60ff6 admin admin] The token cache is being invalidate because project 8674202148024dd19bf52004e35e6764 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:50:12.019850 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-03b7f062-3a8f-49b9-b084-f29bc3b60ff6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:12.020148 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-03b7f062-3a8f-49b9-b084-f29bc3b60ff6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:12.022048 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1037/2053] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:50:11 2022] DELETE /identity/v3/projects/8674202148024dd19bf52004e35e6764 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:12.026783 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1609541-675d-44a1-9b15-76bf439db314 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:12.027185 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1609541-675d-44a1-9b15-76bf439db314 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:12.027466 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1609541-675d-44a1-9b15-76bf439db314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:12.051254 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-f1609541-675d-44a1-9b15-76bf439db314 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:12.052571 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-f1609541-675d-44a1-9b15-76bf439db314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:12.118541 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1017/2054] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:50:12.127587 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43ca1e-d23b-4048-94b7-001d93899cdf None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:12.127928 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43ca1e-d23b-4048-94b7-001d93899cdf None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:12.128208 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43ca1e-d23b-4048-94b7-001d93899cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:12.151167 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1609541-675d-44a1-9b15-76bf439db314 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:12.153529 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-f1609541-675d-44a1-9b15-76bf439db314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:12.156567 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-ed43ca1e-d23b-4048-94b7-001d93899cdf None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:12.157807 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-ed43ca1e-d23b-4048-94b7-001d93899cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:12.166955 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1609541-675d-44a1-9b15-76bf439db314 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:12.168178 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1038/2055] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:50:12.171847 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed43ca1e-d23b-4048-94b7-001d93899cdf None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:12.176665 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc72dbab-8f8b-4c8b-958c-d8ac004b45a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:12.180251 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc72dbab-8f8b-4c8b-958c-d8ac004b45a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:12.180251 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc72dbab-8f8b-4c8b-958c-d8ac004b45a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:12.180251 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc72dbab-8f8b-4c8b-958c-d8ac004b45a2 admin admin] PATH_INFO: `/v3/projects/4ec2507572bf4ecba09f797183706e5a` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:12.181681 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc72dbab-8f8b-4c8b-958c-d8ac004b45a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ec2507572bf4ecba09f797183706e5a)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:12.183737 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed43ca1e-d23b-4048-94b7-001d93899cdf None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:12.184286 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc72dbab-8f8b-4c8b-958c-d8ac004b45a2 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:12.189991 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb51b5-ebdd-4f5b-9f2a-aa6ce9749d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:12.192114 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb51b5-ebdd-4f5b-9f2a-aa6ce9749d61 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:12.192418 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb51b5-ebdd-4f5b-9f2a-aa6ce9749d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:12.192692 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb51b5-ebdd-4f5b-9f2a-aa6ce9749d61 admin admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:12.193444 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cb51b5-ebdd-4f5b-9f2a-aa6ce9749d61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:50:12.193759 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cb51b5-ebdd-4f5b-9f2a-aa6ce9749d61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:12.195331 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cb51b5-ebdd-4f5b-9f2a-aa6ce9749d61 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:12.211404 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1039/2056] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:50:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:50:12.212607 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb51b5-ebdd-4f5b-9f2a-aa6ce9749d61 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:12.222235 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb51b5-ebdd-4f5b-9f2a-aa6ce9749d61 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:12.226919 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0250381-3463-460e-a986-6ed98d5f4513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:12.227514 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0250381-3463-460e-a986-6ed98d5f4513 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:12.227781 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0250381-3463-460e-a986-6ed98d5f4513 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:12.228057 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0250381-3463-460e-a986-6ed98d5f4513 admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:12.228697 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0250381-3463-460e-a986-6ed98d5f4513 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:12.231257 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0250381-3463-460e-a986-6ed98d5f4513 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:12.298329 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-bc72dbab-8f8b-4c8b-958c-d8ac004b45a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ec2507572bf4ecba09f797183706e5a', 'request_id': 'req-bc72dbab-8f8b-4c8b-958c-d8ac004b45a2'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:12.349481 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1040/2057] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:50:12 2022] POST /identity/v3/projects => generated 375 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:50:12.353804 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0250381-3463-460e-a986-6ed98d5f4513 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:12.369194 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-bc72dbab-8f8b-4c8b-958c-d8ac004b45a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ec2507572bf4ecba09f797183706e5a', 'request_id': 'req-bc72dbab-8f8b-4c8b-958c-d8ac004b45a2'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:12.375407 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-bc72dbab-8f8b-4c8b-958c-d8ac004b45a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ec2507572bf4ecba09f797183706e5a', 'request_id': 'req-bc72dbab-8f8b-4c8b-958c-d8ac004b45a2'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:12.387001 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-bc72dbab-8f8b-4c8b-958c-d8ac004b45a2 admin admin] The token cache is being invalidate because project 4ec2507572bf4ecba09f797183706e5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:50:12.387299 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-bc72dbab-8f8b-4c8b-958c-d8ac004b45a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:12.387941 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-bc72dbab-8f8b-4c8b-958c-d8ac004b45a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:12.389497 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1018/2058] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:50:12 2022] DELETE /identity/v3/projects/4ec2507572bf4ecba09f797183706e5a => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:50:12.419766 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0250381-3463-460e-a986-6ed98d5f4513 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:12.426517 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36669d-2dea-4c74-a6a9-3635cc872f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:12.427612 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36669d-2dea-4c74-a6a9-3635cc872f92 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:12.427904 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36669d-2dea-4c74-a6a9-3635cc872f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:12.428309 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36669d-2dea-4c74-a6a9-3635cc872f92 admin admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:12.429079 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad36669d-2dea-4c74-a6a9-3635cc872f92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:12.431787 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad36669d-2dea-4c74-a6a9-3635cc872f92 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:12.648557 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1041/2059] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:50:12 2022] POST /identity/v3/users => generated 288 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:50:12.652951 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc72dbab-8f8b-4c8b-958c-d8ac004b45a2 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:12.654466 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-bc72dbab-8f8b-4c8b-958c-d8ac004b45a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:12.732126 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc72dbab-8f8b-4c8b-958c-d8ac004b45a2 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:12.737273 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00748e-cb0a-48cb-aa6e-ecf69a411ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:12.738058 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00748e-cb0a-48cb-aa6e-ecf69a411ea0 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:12.738611 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00748e-cb0a-48cb-aa6e-ecf69a411ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:12.738815 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00748e-cb0a-48cb-aa6e-ecf69a411ea0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:12.739657 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b00748e-cb0a-48cb-aa6e-ecf69a411ea0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:12.755315 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b00748e-cb0a-48cb-aa6e-ecf69a411ea0 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:12.755315 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1019/2060] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:50:12.759915 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36669d-2dea-4c74-a6a9-3635cc872f92 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:12.772828 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36669d-2dea-4c74-a6a9-3635cc872f92 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:12.777658 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ca071-c5c2-4ea5-8427-b8574b2ca406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:12.778736 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ca071-c5c2-4ea5-8427-b8574b2ca406 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:12.779031 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ca071-c5c2-4ea5-8427-b8574b2ca406 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:12.779404 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ca071-c5c2-4ea5-8427-b8574b2ca406 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:12.780079 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ca071-c5c2-4ea5-8427-b8574b2ca406 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:12.781773 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ca071-c5c2-4ea5-8427-b8574b2ca406 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:12.797735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1042/2061] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:50:12.798877 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00748e-cb0a-48cb-aa6e-ecf69a411ea0 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:12.809978 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ca071-c5c2-4ea5-8427-b8574b2ca406 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:12.810148 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00748e-cb0a-48cb-aa6e-ecf69a411ea0 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:12.811162 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-001ca071-c5c2-4ea5-8427-b8574b2ca406 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:12.815111 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d25b9a-6e19-4ca3-be28-38676cbb176a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:12.815824 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d25b9a-6e19-4ca3-be28-38676cbb176a admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:12.816427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d25b9a-6e19-4ca3-be28-38676cbb176a admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:12.816710 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d25b9a-6e19-4ca3-be28-38676cbb176a admin admin] PATH_INFO: `/v3/projects/7e51e4fb4d424c9bb521b3f9e9e0b623/users/80449a09faeb424dad2e3470430584b7/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:12.825310 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ca071-c5c2-4ea5-8427-b8574b2ca406 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:12.830412 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad46acd5-eb70-411a-9764-f840f02fec25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:12.831057 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46acd5-eb70-411a-9764-f840f02fec25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:12.831312 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46acd5-eb70-411a-9764-f840f02fec25 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:12.832093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46acd5-eb70-411a-9764-f840f02fec25 admin admin] PATH_INFO: `/v3/users/53a5334d87b2402196b012ec0059e088` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:12.833886 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad46acd5-eb70-411a-9764-f840f02fec25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53a5334d87b2402196b012ec0059e088)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:12.835061 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d25b9a-6e19-4ca3-be28-38676cbb176a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e51e4fb4d424c9bb521b3f9e9e0b623, user_id=80449a09faeb424dad2e3470430584b7, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:12.835814 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad46acd5-eb70-411a-9764-f840f02fec25 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:12.838045 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d25b9a-6e19-4ca3-be28-38676cbb176a admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:12.941051 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1020/2062] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:50:12 2022] PUT /identity/v3/projects/7e51e4fb4d424c9bb521b3f9e9e0b623/users/80449a09faeb424dad2e3470430584b7/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:12.951624 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d670e31-9ddc-4d78-bb02-14ad335bb0eb None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:12.951925 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d670e31-9ddc-4d78-bb02-14ad335bb0eb None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:12.952181 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d670e31-9ddc-4d78-bb02-14ad335bb0eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:12.980692 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-3d670e31-9ddc-4d78-bb02-14ad335bb0eb None None] MFA Rules not processed for user `80449a09faeb424dad2e3470430584b7`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:12.981965 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-3d670e31-9ddc-4d78-bb02-14ad335bb0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:13.051975 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1021/2063] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:50:13.070395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d670e31-9ddc-4d78-bb02-14ad335bb0eb None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:13.071789 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-3d670e31-9ddc-4d78-bb02-14ad335bb0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:13.137698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ad46acd5-eb70-411a-9764-f840f02fec25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53a5334d87b2402196b012ec0059e088'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:13.169660 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d670e31-9ddc-4d78-bb02-14ad335bb0eb None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:13.174353 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406f9e22-1a7c-44c7-a5b0-43772f66a98c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c2569e7cd774199aa5750fd6a850060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:13.174967 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f9e22-1a7c-44c7-a5b0-43772f66a98c service glance] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:13.175286 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f9e22-1a7c-44c7-a5b0-43772f66a98c service glance] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:13.175596 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f9e22-1a7c-44c7-a5b0-43772f66a98c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:13.185516 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406f9e22-1a7c-44c7-a5b0-43772f66a98c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:13.188076 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406f9e22-1a7c-44c7-a5b0-43772f66a98c service glance] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:13.199778 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1022/2064] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:50:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:50:13.207367 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ad46acd5-eb70-411a-9764-f840f02fec25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53a5334d87b2402196b012ec0059e088'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:13.223210 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ad46acd5-eb70-411a-9764-f840f02fec25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53a5334d87b2402196b012ec0059e088'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:13.229070 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ad46acd5-eb70-411a-9764-f840f02fec25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53a5334d87b2402196b012ec0059e088'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:13.229774 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ad46acd5-eb70-411a-9764-f840f02fec25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53a5334d87b2402196b012ec0059e088'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:13.236074 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1043/2065] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:50:12 2022] DELETE /identity/v3/users/53a5334d87b2402196b012ec0059e088 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:50:13.241238 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2d600-62ae-4aef-8df2-2c40e796a5d2 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:13.241600 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2d600-62ae-4aef-8df2-2c40e796a5d2 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:13.241873 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2d600-62ae-4aef-8df2-2c40e796a5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:13.262549 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-c5a2d600-62ae-4aef-8df2-2c40e796a5d2 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:13.263758 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-c5a2d600-62ae-4aef-8df2-2c40e796a5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:13.326973 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1023/2066] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:50:13.338384 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad46acd5-eb70-411a-9764-f840f02fec25 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:13.339794 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-ad46acd5-eb70-411a-9764-f840f02fec25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:13.401544 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad46acd5-eb70-411a-9764-f840f02fec25 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:13.405540 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ef6a5b-c82e-4518-b09d-b5dbcd32b4f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:13.406216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef6a5b-c82e-4518-b09d-b5dbcd32b4f1 service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:13.406502 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef6a5b-c82e-4518-b09d-b5dbcd32b4f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:13.406779 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef6a5b-c82e-4518-b09d-b5dbcd32b4f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:13.418131 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ef6a5b-c82e-4518-b09d-b5dbcd32b4f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:13.419631 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ef6a5b-c82e-4518-b09d-b5dbcd32b4f1 service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:13.430977 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1044/2067] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:50:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:50:13.624510 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:13.838495 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:13.897763 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2d600-62ae-4aef-8df2-2c40e796a5d2 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:13.899156 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-c5a2d600-62ae-4aef-8df2-2c40e796a5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:13.932249 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ef6a5b-c82e-4518-b09d-b5dbcd32b4f1 service neutron] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:13.933934 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-51ef6a5b-c82e-4518-b09d-b5dbcd32b4f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:13.951664 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ef6a5b-c82e-4518-b09d-b5dbcd32b4f1 service neutron] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:13.969319 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb245c-5e03-4068-8a5f-d69ec8dc88c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:13.969982 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb245c-5e03-4068-8a5f-d69ec8dc88c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:13.970261 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb245c-5e03-4068-8a5f-d69ec8dc88c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:13.970534 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb245c-5e03-4068-8a5f-d69ec8dc88c5 admin admin] PATH_INFO: `/v3/projects/e3ad4fc405c5475492a194d79dedc957` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:13.971740 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadb245c-5e03-4068-8a5f-d69ec8dc88c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3ad4fc405c5475492a194d79dedc957)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:13.973976 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadb245c-5e03-4068-8a5f-d69ec8dc88c5 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:13.976398 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2d600-62ae-4aef-8df2-2c40e796a5d2 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:13.987675 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62825e17-9bfc-4749-97de-2a79c4c2d767 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:13.990772 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62825e17-9bfc-4749-97de-2a79c4c2d767 service cinder] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:13.990772 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62825e17-9bfc-4749-97de-2a79c4c2d767 service cinder] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:13.990772 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62825e17-9bfc-4749-97de-2a79c4c2d767 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:13.991891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-62825e17-9bfc-4749-97de-2a79c4c2d767 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:14.054759 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62825e17-9bfc-4749-97de-2a79c4c2d767 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:14.056192 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62825e17-9bfc-4749-97de-2a79c4c2d767 service cinder] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:14.065465 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1024/2068] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:50:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:50:14.178117 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-cadb245c-5e03-4068-8a5f-d69ec8dc88c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3ad4fc405c5475492a194d79dedc957', 'request_id': 'req-cadb245c-5e03-4068-8a5f-d69ec8dc88c5'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:14.188026 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-cadb245c-5e03-4068-8a5f-d69ec8dc88c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3ad4fc405c5475492a194d79dedc957', 'request_id': 'req-cadb245c-5e03-4068-8a5f-d69ec8dc88c5'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:14.205200 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-cadb245c-5e03-4068-8a5f-d69ec8dc88c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3ad4fc405c5475492a194d79dedc957', 'request_id': 'req-cadb245c-5e03-4068-8a5f-d69ec8dc88c5'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:14.324291 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-cadb245c-5e03-4068-8a5f-d69ec8dc88c5 admin admin] The token cache is being invalidate because project e3ad4fc405c5475492a194d79dedc957 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:50:14.327047 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-cadb245c-5e03-4068-8a5f-d69ec8dc88c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:14.327047 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-cadb245c-5e03-4068-8a5f-d69ec8dc88c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:14.327842 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1045/2069] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:50:13 2022] DELETE /identity/v3/projects/e3ad4fc405c5475492a194d79dedc957 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:50:14.333010 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62825e17-9bfc-4749-97de-2a79c4c2d767 service cinder] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:14.345278 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62825e17-9bfc-4749-97de-2a79c4c2d767 service cinder] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:14.407194 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e626b4-6981-485d-9059-b83dc7f0206d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:14.407929 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e626b4-6981-485d-9059-b83dc7f0206d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:14.408508 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e626b4-6981-485d-9059-b83dc7f0206d admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:14.409006 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e626b4-6981-485d-9059-b83dc7f0206d admin admin] PATH_INFO: `/v3/users/321c5f768791446fbb20bf0d8340d9d9` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:14.410528 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e626b4-6981-485d-9059-b83dc7f0206d admin admin] RBAC: Authorizing `identity:delete_user(user_id=321c5f768791446fbb20bf0d8340d9d9)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:14.413252 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e626b4-6981-485d-9059-b83dc7f0206d admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:14.718724 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-01e626b4-6981-485d-9059-b83dc7f0206d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '321c5f768791446fbb20bf0d8340d9d9'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:14.733109 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-01e626b4-6981-485d-9059-b83dc7f0206d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '321c5f768791446fbb20bf0d8340d9d9'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:14.743579 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-01e626b4-6981-485d-9059-b83dc7f0206d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '321c5f768791446fbb20bf0d8340d9d9'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:14.835512 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-01e626b4-6981-485d-9059-b83dc7f0206d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '321c5f768791446fbb20bf0d8340d9d9'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:14.836372 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-01e626b4-6981-485d-9059-b83dc7f0206d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '321c5f768791446fbb20bf0d8340d9d9'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:14.836562 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=83,keystone:DELETE=12,keystone:INSERT=5 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:14.843774 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1025/2070] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:50:14 2022] DELETE /identity/v3/users/321c5f768791446fbb20bf0d8340d9d9 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:15.450858 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:15.452914 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb245c-5e03-4068-8a5f-d69ec8dc88c5 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:15.454548 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-cadb245c-5e03-4068-8a5f-d69ec8dc88c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:15.470545 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb245c-5e03-4068-8a5f-d69ec8dc88c5 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:15.538292 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a97de-fb7a-4e8d-81f5-503ccf4acbb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:15.538857 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a97de-fb7a-4e8d-81f5-503ccf4acbb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:15.539211 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a97de-fb7a-4e8d-81f5-503ccf4acbb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:15.539415 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a97de-fb7a-4e8d-81f5-503ccf4acbb4 admin admin] PATH_INFO: `/v3/projects/ed3f681f20d1495b939a9a7af243fcce` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:15.540768 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333a97de-fb7a-4e8d-81f5-503ccf4acbb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed3f681f20d1495b939a9a7af243fcce)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:15.542535 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333a97de-fb7a-4e8d-81f5-503ccf4acbb4 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:15.793888 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-333a97de-fb7a-4e8d-81f5-503ccf4acbb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed3f681f20d1495b939a9a7af243fcce', 'request_id': 'req-333a97de-fb7a-4e8d-81f5-503ccf4acbb4'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:15.799368 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-333a97de-fb7a-4e8d-81f5-503ccf4acbb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed3f681f20d1495b939a9a7af243fcce', 'request_id': 'req-333a97de-fb7a-4e8d-81f5-503ccf4acbb4'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:15.806064 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-333a97de-fb7a-4e8d-81f5-503ccf4acbb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed3f681f20d1495b939a9a7af243fcce', 'request_id': 'req-333a97de-fb7a-4e8d-81f5-503ccf4acbb4'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:15.929223 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-333a97de-fb7a-4e8d-81f5-503ccf4acbb4 admin admin] The token cache is being invalidate because project ed3f681f20d1495b939a9a7af243fcce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:50:15.929554 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-333a97de-fb7a-4e8d-81f5-503ccf4acbb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:15.930200 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-333a97de-fb7a-4e8d-81f5-503ccf4acbb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:15.931710 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1046/2071] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:50:15 2022] DELETE /identity/v3/projects/ed3f681f20d1495b939a9a7af243fcce => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:15.935849 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e626b4-6981-485d-9059-b83dc7f0206d admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:15.937398 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-01e626b4-6981-485d-9059-b83dc7f0206d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:15.956347 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e626b4-6981-485d-9059-b83dc7f0206d admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:16.019078 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c805ff-924d-47b0-ab92-84f6e7079f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:16.020253 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c805ff-924d-47b0-ab92-84f6e7079f73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:16.020538 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c805ff-924d-47b0-ab92-84f6e7079f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:16.020852 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c805ff-924d-47b0-ab92-84f6e7079f73 admin admin] PATH_INFO: `/v3/users/f1afc5154fe6487580a8449a37b92f02` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:16.021986 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c805ff-924d-47b0-ab92-84f6e7079f73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1afc5154fe6487580a8449a37b92f02)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:16.024098 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c805ff-924d-47b0-ab92-84f6e7079f73 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:16.086799 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a97de-fb7a-4e8d-81f5-503ccf4acbb4 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:16.090484 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-333a97de-fb7a-4e8d-81f5-503ccf4acbb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:16.149657 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a97de-fb7a-4e8d-81f5-503ccf4acbb4 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:16.153885 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decd8341-1c28-49d1-8d06-9467eba71ec1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:16.154767 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd8341-1c28-49d1-8d06-9467eba71ec1 service cinder] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:16.154953 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd8341-1c28-49d1-8d06-9467eba71ec1 service cinder] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:16.155321 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd8341-1c28-49d1-8d06-9467eba71ec1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:16.157268 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-decd8341-1c28-49d1-8d06-9467eba71ec1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:16.246620 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decd8341-1c28-49d1-8d06-9467eba71ec1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:16.246620 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decd8341-1c28-49d1-8d06-9467eba71ec1 service cinder] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:16.247187 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-41c805ff-924d-47b0-ab92-84f6e7079f73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1afc5154fe6487580a8449a37b92f02'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:16.251908 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1047/2072] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:50:16 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:50:16.268179 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-41c805ff-924d-47b0-ab92-84f6e7079f73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1afc5154fe6487580a8449a37b92f02'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:16.276668 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-41c805ff-924d-47b0-ab92-84f6e7079f73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1afc5154fe6487580a8449a37b92f02'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:16.350783 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-41c805ff-924d-47b0-ab92-84f6e7079f73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1afc5154fe6487580a8449a37b92f02'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:16.352390 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-41c805ff-924d-47b0-ab92-84f6e7079f73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1afc5154fe6487580a8449a37b92f02'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:16.358117 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1026/2073] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:50:15 2022] DELETE /identity/v3/users/f1afc5154fe6487580a8449a37b92f02 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:16.862488 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decd8341-1c28-49d1-8d06-9467eba71ec1 service cinder] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:16.864605 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-decd8341-1c28-49d1-8d06-9467eba71ec1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:16.878415 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decd8341-1c28-49d1-8d06-9467eba71ec1 service cinder] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:16.936547 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a15dde2-2b54-482b-860f-306dc0c75145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:16.937199 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a15dde2-2b54-482b-860f-306dc0c75145 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:16.937477 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a15dde2-2b54-482b-860f-306dc0c75145 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:16.937746 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a15dde2-2b54-482b-860f-306dc0c75145 admin admin] PATH_INFO: `/v3/projects/5b433a2329974d1aa6c524201ce21af4` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:16.938827 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a15dde2-2b54-482b-860f-306dc0c75145 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b433a2329974d1aa6c524201ce21af4)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:16.940743 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a15dde2-2b54-482b-860f-306dc0c75145 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:17.094658 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-3a15dde2-2b54-482b-860f-306dc0c75145 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b433a2329974d1aa6c524201ce21af4', 'request_id': 'req-3a15dde2-2b54-482b-860f-306dc0c75145'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:17.100953 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-3a15dde2-2b54-482b-860f-306dc0c75145 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b433a2329974d1aa6c524201ce21af4', 'request_id': 'req-3a15dde2-2b54-482b-860f-306dc0c75145'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:17.107459 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-3a15dde2-2b54-482b-860f-306dc0c75145 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b433a2329974d1aa6c524201ce21af4', 'request_id': 'req-3a15dde2-2b54-482b-860f-306dc0c75145'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:17.178575 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-3a15dde2-2b54-482b-860f-306dc0c75145 admin admin] The token cache is being invalidate because project 5b433a2329974d1aa6c524201ce21af4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:50:17.178869 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-3a15dde2-2b54-482b-860f-306dc0c75145 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:17.179498 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-3a15dde2-2b54-482b-860f-306dc0c75145 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:17.181027 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1048/2074] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:50:16 2022] DELETE /identity/v3/projects/5b433a2329974d1aa6c524201ce21af4 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:17.187869 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa494534-f587-45e4-8090-cadf3262df8c None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:17.188345 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa494534-f587-45e4-8090-cadf3262df8c None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:17.188447 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa494534-f587-45e4-8090-cadf3262df8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:17.209878 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-aa494534-f587-45e4-8090-cadf3262df8c None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:17.211142 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-aa494534-f587-45e4-8090-cadf3262df8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:17.276616 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1027/2075] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:50:17.277810 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:17.284491 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b50749-624e-4dd3-b00a-0427a8d16402 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:17.284911 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b50749-624e-4dd3-b00a-0427a8d16402 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:17.293279 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b50749-624e-4dd3-b00a-0427a8d16402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:17.310716 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-a2b50749-624e-4dd3-b00a-0427a8d16402 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:17.311880 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-a2b50749-624e-4dd3-b00a-0427a8d16402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:17.320142 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1049/2076] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:50:17.347355 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa494534-f587-45e4-8090-cadf3262df8c None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:17.364850 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa494534-f587-45e4-8090-cadf3262df8c None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:17.370722 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebb9c14-9889-44b1-a401-64e271514a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:17.371353 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb9c14-9889-44b1-a401-64e271514a53 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:17.371630 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb9c14-9889-44b1-a401-64e271514a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:17.371978 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb9c14-9889-44b1-a401-64e271514a53 admin admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:17.372989 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebb9c14-9889-44b1-a401-64e271514a53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:50:17.373343 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebb9c14-9889-44b1-a401-64e271514a53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:17.375525 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebb9c14-9889-44b1-a401-64e271514a53 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:17.388991 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1028/2077] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:50:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:50:17.393718 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b50749-624e-4dd3-b00a-0427a8d16402 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:17.405743 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b50749-624e-4dd3-b00a-0427a8d16402 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:17.418036 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8e0839-f62d-4144-bfc1-ff3070f75ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:17.418865 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e0839-f62d-4144-bfc1-ff3070f75ded admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:17.419211 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e0839-f62d-4144-bfc1-ff3070f75ded admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:17.419700 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e0839-f62d-4144-bfc1-ff3070f75ded admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:17.420330 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8e0839-f62d-4144-bfc1-ff3070f75ded admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:17.422560 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8e0839-f62d-4144-bfc1-ff3070f75ded admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:17.531735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1050/2078] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:50:17 2022] POST /identity/v3/projects => generated 381 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:50:17.536106 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebb9c14-9889-44b1-a401-64e271514a53 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:17.548413 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebb9c14-9889-44b1-a401-64e271514a53 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:17.609859 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5729b189-2849-4cff-a4ab-28be04d76674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:17.610364 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729b189-2849-4cff-a4ab-28be04d76674 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:17.610642 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729b189-2849-4cff-a4ab-28be04d76674 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:17.610912 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729b189-2849-4cff-a4ab-28be04d76674 admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:17.611546 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5729b189-2849-4cff-a4ab-28be04d76674 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:17.614751 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5729b189-2849-4cff-a4ab-28be04d76674 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:17.809647 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1029/2079] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:50:17 2022] POST /identity/v3/users => generated 291 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:50:17.813844 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8e0839-f62d-4144-bfc1-ff3070f75ded admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:17.824605 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8e0839-f62d-4144-bfc1-ff3070f75ded admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:17.829150 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc91f1-d132-4b71-bbc4-9bb6b06b83a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:17.829742 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc91f1-d132-4b71-bbc4-9bb6b06b83a0 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:17.830274 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc91f1-d132-4b71-bbc4-9bb6b06b83a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:17.830555 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc91f1-d132-4b71-bbc4-9bb6b06b83a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:17.831187 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efc91f1-d132-4b71-bbc4-9bb6b06b83a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:17.833302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efc91f1-d132-4b71-bbc4-9bb6b06b83a0 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:17.843747 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1051/2080] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:50:17.848526 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5729b189-2849-4cff-a4ab-28be04d76674 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:17.859248 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5729b189-2849-4cff-a4ab-28be04d76674 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:17.865710 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196e5cec-4721-4f05-93fc-2e124a228c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:17.866801 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e5cec-4721-4f05-93fc-2e124a228c2a admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:17.867266 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e5cec-4721-4f05-93fc-2e124a228c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:17.867603 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e5cec-4721-4f05-93fc-2e124a228c2a admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:17.869377 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196e5cec-4721-4f05-93fc-2e124a228c2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:17.873696 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196e5cec-4721-4f05-93fc-2e124a228c2a admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:17.891296 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1030/2081] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:17 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:50:17.895649 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc91f1-d132-4b71-bbc4-9bb6b06b83a0 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:17.905811 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc91f1-d132-4b71-bbc4-9bb6b06b83a0 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:17.911798 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fe772a-054d-4827-9e10-ef8c53805d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:17.912620 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe772a-054d-4827-9e10-ef8c53805d1d admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:17.912944 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe772a-054d-4827-9e10-ef8c53805d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:17.913221 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe772a-054d-4827-9e10-ef8c53805d1d admin admin] PATH_INFO: `/v3/projects/745f9070761643e599464fa24cdfb2d8/users/6430def1a8cf4e76adde463b37ce38e9/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:17.919083 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:17.929747 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fe772a-054d-4827-9e10-ef8c53805d1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=745f9070761643e599464fa24cdfb2d8, user_id=6430def1a8cf4e76adde463b37ce38e9, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:17.932430 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fe772a-054d-4827-9e10-ef8c53805d1d admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:18.039877 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1052/2082] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:50:17 2022] PUT /identity/v3/projects/745f9070761643e599464fa24cdfb2d8/users/6430def1a8cf4e76adde463b37ce38e9/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:18.054490 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae859a6-9554-4e24-9d88-cd8d8c009eaa None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:18.054984 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae859a6-9554-4e24-9d88-cd8d8c009eaa None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:18.055303 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae859a6-9554-4e24-9d88-cd8d8c009eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:18.088204 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-2ae859a6-9554-4e24-9d88-cd8d8c009eaa None None] MFA Rules not processed for user `6430def1a8cf4e76adde463b37ce38e9`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:18.089559 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-2ae859a6-9554-4e24-9d88-cd8d8c009eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:18.157367 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1031/2083] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:50:18.162872 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fe772a-054d-4827-9e10-ef8c53805d1d admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:18.256807 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fe772a-054d-4827-9e10-ef8c53805d1d admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:18.264123 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7eaef4-047e-4f4d-bd64-92c4d98029af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:18.265270 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7eaef4-047e-4f4d-bd64-92c4d98029af admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:18.265818 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7eaef4-047e-4f4d-bd64-92c4d98029af admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:18.266307 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7eaef4-047e-4f4d-bd64-92c4d98029af admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:18.267199 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7eaef4-047e-4f4d-bd64-92c4d98029af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:18.270750 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7eaef4-047e-4f4d-bd64-92c4d98029af admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:18.335764 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:18.381401 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1053/2084] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:50:18 2022] POST /identity/v3/projects => generated 381 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:50:18.387130 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae859a6-9554-4e24-9d88-cd8d8c009eaa None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:18.398004 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae859a6-9554-4e24-9d88-cd8d8c009eaa None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:18.465539 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c82ec21-b0cc-4d7b-9712-40965e3e9057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:18.465539 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c82ec21-b0cc-4d7b-9712-40965e3e9057 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:18.465539 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c82ec21-b0cc-4d7b-9712-40965e3e9057 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:18.465539 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c82ec21-b0cc-4d7b-9712-40965e3e9057 admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:18.466318 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c82ec21-b0cc-4d7b-9712-40965e3e9057 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:18.466318 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c82ec21-b0cc-4d7b-9712-40965e3e9057 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:18.586731 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1032/2085] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:50:18 2022] POST /identity/v3/users => generated 297 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:50:18.590909 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7eaef4-047e-4f4d-bd64-92c4d98029af admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:18.604072 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7eaef4-047e-4f4d-bd64-92c4d98029af admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:18.618218 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fddbda2-24a6-4faf-b3e4-d5329be12997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:18.619111 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fddbda2-24a6-4faf-b3e4-d5329be12997 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:18.619398 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fddbda2-24a6-4faf-b3e4-d5329be12997 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:18.619673 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fddbda2-24a6-4faf-b3e4-d5329be12997 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:18.620420 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fddbda2-24a6-4faf-b3e4-d5329be12997 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:18.622063 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fddbda2-24a6-4faf-b3e4-d5329be12997 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:18.634141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1054/2086] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:50:18.638236 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c82ec21-b0cc-4d7b-9712-40965e3e9057 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:18.649584 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c82ec21-b0cc-4d7b-9712-40965e3e9057 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:18.654695 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e1e5c-294b-4e72-8f40-0c2f95ae8281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:18.661138 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e1e5c-294b-4e72-8f40-0c2f95ae8281 admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:18.661138 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e1e5c-294b-4e72-8f40-0c2f95ae8281 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:18.661138 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e1e5c-294b-4e72-8f40-0c2f95ae8281 admin admin] PATH_INFO: `/v3/projects/4ddde0870e404d8fab2f3d6c2a0f9ddb/users/9213344a6c764bb99c9dbaa2318b2c4f/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:18.672219 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701e1e5c-294b-4e72-8f40-0c2f95ae8281 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ddde0870e404d8fab2f3d6c2a0f9ddb, user_id=9213344a6c764bb99c9dbaa2318b2c4f, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:18.675200 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701e1e5c-294b-4e72-8f40-0c2f95ae8281 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:18.795978 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1033/2087] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:50:18 2022] PUT /identity/v3/projects/4ddde0870e404d8fab2f3d6c2a0f9ddb/users/9213344a6c764bb99c9dbaa2318b2c4f/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:18.800175 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fddbda2-24a6-4faf-b3e4-d5329be12997 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:18.822857 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fddbda2-24a6-4faf-b3e4-d5329be12997 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:18.889738 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:18.899896 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27ac595-93f3-4be8-9d60-ca801d5b4c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:18.900617 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27ac595-93f3-4be8-9d60-ca801d5b4c77 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:18.900949 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27ac595-93f3-4be8-9d60-ca801d5b4c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:18.901227 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27ac595-93f3-4be8-9d60-ca801d5b4c77 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:18.901877 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27ac595-93f3-4be8-9d60-ca801d5b4c77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:18.903638 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27ac595-93f3-4be8-9d60-ca801d5b4c77 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:18.914767 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1055/2088] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:50:18.919038 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e1e5c-294b-4e72-8f40-0c2f95ae8281 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:18.930334 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e1e5c-294b-4e72-8f40-0c2f95ae8281 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:18.935361 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdaa89f-8e5f-4c43-a942-51158771483d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:18.935965 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdaa89f-8e5f-4c43-a942-51158771483d admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:18.936421 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdaa89f-8e5f-4c43-a942-51158771483d admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:18.936561 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdaa89f-8e5f-4c43-a942-51158771483d admin admin] PATH_INFO: `/v3/projects/4ddde0870e404d8fab2f3d6c2a0f9ddb/users/9213344a6c764bb99c9dbaa2318b2c4f/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:18.938542 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdaa89f-8e5f-4c43-a942-51158771483d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ddde0870e404d8fab2f3d6c2a0f9ddb, user_id=9213344a6c764bb99c9dbaa2318b2c4f, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:18.942158 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdaa89f-8e5f-4c43-a942-51158771483d admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:18.955605 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1034/2089] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:50:18 2022] PUT /identity/v3/projects/4ddde0870e404d8fab2f3d6c2a0f9ddb/users/9213344a6c764bb99c9dbaa2318b2c4f/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:50:18.966792 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf5e42-be75-40c5-8ffc-6803ea855a69 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:18.967078 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf5e42-be75-40c5-8ffc-6803ea855a69 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:18.967351 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf5e42-be75-40c5-8ffc-6803ea855a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:18.995287 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-badf5e42-be75-40c5-8ffc-6803ea855a69 None None] MFA Rules not processed for user `9213344a6c764bb99c9dbaa2318b2c4f`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:18.996512 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-badf5e42-be75-40c5-8ffc-6803ea855a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:19.079510 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1056/2090] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:18 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:50:19.092270 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdaa89f-8e5f-4c43-a942-51158771483d admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:19.093995 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-6cdaa89f-8e5f-4c43-a942-51158771483d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:19.158169 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdaa89f-8e5f-4c43-a942-51158771483d admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:19.162392 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078621a3-54e5-413d-8bf4-97382fd53b78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:19.163015 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078621a3-54e5-413d-8bf4-97382fd53b78 service cinder] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:19.163300 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078621a3-54e5-413d-8bf4-97382fd53b78 service cinder] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:19.163573 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078621a3-54e5-413d-8bf4-97382fd53b78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:19.174254 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078621a3-54e5-413d-8bf4-97382fd53b78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:19.176048 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078621a3-54e5-413d-8bf4-97382fd53b78 service cinder] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:19.188521 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1035/2091] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:50:19 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:50:19.512820 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:19.839748 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badf5e42-be75-40c5-8ffc-6803ea855a69 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:19.905794 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badf5e42-be75-40c5-8ffc-6803ea855a69 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:19.914043 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeab25e-a409-4214-bd3a-52d1192bea78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:19.914899 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeab25e-a409-4214-bd3a-52d1192bea78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:19.915599 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeab25e-a409-4214-bd3a-52d1192bea78 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:19.916042 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeab25e-a409-4214-bd3a-52d1192bea78 admin admin] PATH_INFO: `/v3/users/6430def1a8cf4e76adde463b37ce38e9` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:19.917434 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeab25e-a409-4214-bd3a-52d1192bea78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6430def1a8cf4e76adde463b37ce38e9)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:19.919706 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeab25e-a409-4214-bd3a-52d1192bea78 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:20.055471 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:20.144098 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-1aeab25e-a409-4214-bd3a-52d1192bea78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6430def1a8cf4e76adde463b37ce38e9'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:20.248956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-1aeab25e-a409-4214-bd3a-52d1192bea78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6430def1a8cf4e76adde463b37ce38e9'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:20.263142 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-1aeab25e-a409-4214-bd3a-52d1192bea78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6430def1a8cf4e76adde463b37ce38e9'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:20.270660 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-1aeab25e-a409-4214-bd3a-52d1192bea78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6430def1a8cf4e76adde463b37ce38e9'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:20.271360 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-1aeab25e-a409-4214-bd3a-52d1192bea78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6430def1a8cf4e76adde463b37ce38e9'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:20.277300 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1057/2092] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:50:19 2022] DELETE /identity/v3/users/6430def1a8cf4e76adde463b37ce38e9 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:20.281797 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfaf63f-d11a-4ce1-a80a-ce2881babb07 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:20.282079 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfaf63f-d11a-4ce1-a80a-ce2881babb07 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:20.282350 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfaf63f-d11a-4ce1-a80a-ce2881babb07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:20.314161 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-bdfaf63f-d11a-4ce1-a80a-ce2881babb07 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:20.316682 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-bdfaf63f-d11a-4ce1-a80a-ce2881babb07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:20.400753 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1036/2093] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:50:20.410380 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeab25e-a409-4214-bd3a-52d1192bea78 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:20.412125 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-1aeab25e-a409-4214-bd3a-52d1192bea78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:20.479287 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeab25e-a409-4214-bd3a-52d1192bea78 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:20.482928 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31161185-3d00-47b4-b755-4745d8c74a76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:20.483571 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31161185-3d00-47b4-b755-4745d8c74a76 service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:20.483856 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31161185-3d00-47b4-b755-4745d8c74a76 service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:20.484129 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31161185-3d00-47b4-b755-4745d8c74a76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:20.494685 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31161185-3d00-47b4-b755-4745d8c74a76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:20.496162 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31161185-3d00-47b4-b755-4745d8c74a76 service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:20.507705 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1058/2094] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:50:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:50:20.512212 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:20.792069 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:20.979741 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfaf63f-d11a-4ce1-a80a-ce2881babb07 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:20.979741 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-bdfaf63f-d11a-4ce1-a80a-ce2881babb07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:20.991949 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfaf63f-d11a-4ce1-a80a-ce2881babb07 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:20.998099 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d7642-3e77-466d-b8ed-b436c6cb8b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:20.998945 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d7642-3e77-466d-b8ed-b436c6cb8b24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:20.999318 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d7642-3e77-466d-b8ed-b436c6cb8b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:20.999590 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d7642-3e77-466d-b8ed-b436c6cb8b24 admin admin] PATH_INFO: `/v3/projects/745f9070761643e599464fa24cdfb2d8` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:21.000829 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d7642-3e77-466d-b8ed-b436c6cb8b24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=745f9070761643e599464fa24cdfb2d8)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:21.002856 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d7642-3e77-466d-b8ed-b436c6cb8b24 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:21.140660 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-b61d7642-3e77-466d-b8ed-b436c6cb8b24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '745f9070761643e599464fa24cdfb2d8', 'request_id': 'req-b61d7642-3e77-466d-b8ed-b436c6cb8b24'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:21.264150 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-b61d7642-3e77-466d-b8ed-b436c6cb8b24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '745f9070761643e599464fa24cdfb2d8', 'request_id': 'req-b61d7642-3e77-466d-b8ed-b436c6cb8b24'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:21.271681 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-b61d7642-3e77-466d-b8ed-b436c6cb8b24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '745f9070761643e599464fa24cdfb2d8', 'request_id': 'req-b61d7642-3e77-466d-b8ed-b436c6cb8b24'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:21.282769 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-b61d7642-3e77-466d-b8ed-b436c6cb8b24 admin admin] The token cache is being invalidate because project 745f9070761643e599464fa24cdfb2d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:50:21.283059 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-b61d7642-3e77-466d-b8ed-b436c6cb8b24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:21.283702 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-b61d7642-3e77-466d-b8ed-b436c6cb8b24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:21.285950 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1037/2095] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:50:20 2022] DELETE /identity/v3/projects/745f9070761643e599464fa24cdfb2d8 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:21.289893 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31161185-3d00-47b4-b755-4745d8c74a76 service neutron] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:21.299785 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31161185-3d00-47b4-b755-4745d8c74a76 service neutron] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:21.353720 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:21.359545 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985cb895-325b-4bec-92aa-f24d3086f84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:21.360172 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985cb895-325b-4bec-92aa-f24d3086f84b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:21.360454 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985cb895-325b-4bec-92aa-f24d3086f84b admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:21.360746 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985cb895-325b-4bec-92aa-f24d3086f84b admin admin] PATH_INFO: `/v3/users/9213344a6c764bb99c9dbaa2318b2c4f` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:21.361993 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985cb895-325b-4bec-92aa-f24d3086f84b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9213344a6c764bb99c9dbaa2318b2c4f)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:21.364064 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985cb895-325b-4bec-92aa-f24d3086f84b admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:21.579274 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-985cb895-325b-4bec-92aa-f24d3086f84b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9213344a6c764bb99c9dbaa2318b2c4f'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:21.649230 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-985cb895-325b-4bec-92aa-f24d3086f84b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9213344a6c764bb99c9dbaa2318b2c4f'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:21.662219 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-985cb895-325b-4bec-92aa-f24d3086f84b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9213344a6c764bb99c9dbaa2318b2c4f'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:21.669060 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-985cb895-325b-4bec-92aa-f24d3086f84b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9213344a6c764bb99c9dbaa2318b2c4f'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:21.669720 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-985cb895-325b-4bec-92aa-f24d3086f84b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9213344a6c764bb99c9dbaa2318b2c4f'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:21.676677 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1059/2096] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:50:21 2022] DELETE /identity/v3/users/9213344a6c764bb99c9dbaa2318b2c4f => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:22.119718 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d7642-3e77-466d-b8ed-b436c6cb8b24 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:22.121984 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-b61d7642-3e77-466d-b8ed-b436c6cb8b24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:22.135646 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d7642-3e77-466d-b8ed-b436c6cb8b24 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:22.209648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f97134d-0283-464b-8cee-c5c9e923956f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:22.210487 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97134d-0283-464b-8cee-c5c9e923956f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:22.210867 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97134d-0283-464b-8cee-c5c9e923956f admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:22.211263 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97134d-0283-464b-8cee-c5c9e923956f admin admin] PATH_INFO: `/v3/projects/4ddde0870e404d8fab2f3d6c2a0f9ddb` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:22.212476 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f97134d-0283-464b-8cee-c5c9e923956f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ddde0870e404d8fab2f3d6c2a0f9ddb)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:22.214655 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f97134d-0283-464b-8cee-c5c9e923956f admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:22.326056 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:22.341338 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-7f97134d-0283-464b-8cee-c5c9e923956f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ddde0870e404d8fab2f3d6c2a0f9ddb', 'request_id': 'req-7f97134d-0283-464b-8cee-c5c9e923956f'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:22.403681 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-7f97134d-0283-464b-8cee-c5c9e923956f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ddde0870e404d8fab2f3d6c2a0f9ddb', 'request_id': 'req-7f97134d-0283-464b-8cee-c5c9e923956f'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:22.409714 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-7f97134d-0283-464b-8cee-c5c9e923956f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ddde0870e404d8fab2f3d6c2a0f9ddb', 'request_id': 'req-7f97134d-0283-464b-8cee-c5c9e923956f'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:22.422380 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-7f97134d-0283-464b-8cee-c5c9e923956f admin admin] The token cache is being invalidate because project 4ddde0870e404d8fab2f3d6c2a0f9ddb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:50:22.422678 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-7f97134d-0283-464b-8cee-c5c9e923956f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:22.423508 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-7f97134d-0283-464b-8cee-c5c9e923956f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:22.427655 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1038/2097] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:50:22 2022] DELETE /identity/v3/projects/4ddde0870e404d8fab2f3d6c2a0f9ddb => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:22.432953 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e522389-f43e-4e42-a1b7-f71f6410bcfd None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:22.433948 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e522389-f43e-4e42-a1b7-f71f6410bcfd None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:22.434532 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e522389-f43e-4e42-a1b7-f71f6410bcfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:22.455981 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-5e522389-f43e-4e42-a1b7-f71f6410bcfd None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:22.457731 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-5e522389-f43e-4e42-a1b7-f71f6410bcfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:22.516010 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1060/2098] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:50:22.524751 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570e749-54c7-4c62-b68a-8912e9354e8a None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:22.525060 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570e749-54c7-4c62-b68a-8912e9354e8a None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:22.525334 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570e749-54c7-4c62-b68a-8912e9354e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:22.548553 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-9570e749-54c7-4c62-b68a-8912e9354e8a None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:22.550112 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-9570e749-54c7-4c62-b68a-8912e9354e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:22.559541 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1039/2099] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:50:22.563942 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e522389-f43e-4e42-a1b7-f71f6410bcfd None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:22.574007 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e522389-f43e-4e42-a1b7-f71f6410bcfd None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:22.578878 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ac4564-7581-4fcb-84ef-78ac1072ab40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:22.579481 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac4564-7581-4fcb-84ef-78ac1072ab40 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:22.579760 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac4564-7581-4fcb-84ef-78ac1072ab40 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:22.580032 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac4564-7581-4fcb-84ef-78ac1072ab40 admin admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:22.581012 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ac4564-7581-4fcb-84ef-78ac1072ab40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:50:22.581600 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ac4564-7581-4fcb-84ef-78ac1072ab40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:22.583611 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ac4564-7581-4fcb-84ef-78ac1072ab40 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:22.600298 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1061/2100] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:50:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:50:22.601465 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9570e749-54c7-4c62-b68a-8912e9354e8a None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:22.614505 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9570e749-54c7-4c62-b68a-8912e9354e8a None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:22.635324 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ff57d-9d34-4429-a1f4-d2a6d6c21499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:22.636330 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ff57d-9d34-4429-a1f4-d2a6d6c21499 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:22.636612 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ff57d-9d34-4429-a1f4-d2a6d6c21499 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:22.636939 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ff57d-9d34-4429-a1f4-d2a6d6c21499 admin admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:22.637578 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ff57d-9d34-4429-a1f4-d2a6d6c21499 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:22.639488 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ff57d-9d34-4429-a1f4-d2a6d6c21499 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:22.728147 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1040/2101] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:50:22 2022] POST /identity/v3/projects => generated 375 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:50:22.733731 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ac4564-7581-4fcb-84ef-78ac1072ab40 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:22.755581 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ac4564-7581-4fcb-84ef-78ac1072ab40 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:22.821228 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a7acb-ef4d-4762-90ac-50978dd51e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:22.821839 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a7acb-ef4d-4762-90ac-50978dd51e30 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:22.822120 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a7acb-ef4d-4762-90ac-50978dd51e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:22.822393 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a7acb-ef4d-4762-90ac-50978dd51e30 admin admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:22.823040 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8a7acb-ef4d-4762-90ac-50978dd51e30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:22.824991 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8a7acb-ef4d-4762-90ac-50978dd51e30 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:22.835454 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:22.949749 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1062/2102] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:50:22 2022] POST /identity/v3/users => generated 288 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:50:22.953899 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ff57d-9d34-4429-a1f4-d2a6d6c21499 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:22.964517 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ff57d-9d34-4429-a1f4-d2a6d6c21499 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:22.969195 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a48b742-2815-4059-9317-c4d465635baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:22.969775 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a48b742-2815-4059-9317-c4d465635baf admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:22.970053 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a48b742-2815-4059-9317-c4d465635baf admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:22.970326 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a48b742-2815-4059-9317-c4d465635baf admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:22.970977 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a48b742-2815-4059-9317-c4d465635baf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:22.978333 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a48b742-2815-4059-9317-c4d465635baf admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:22.983497 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1041/2103] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:50:22.987885 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a7acb-ef4d-4762-90ac-50978dd51e30 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:23.002156 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a7acb-ef4d-4762-90ac-50978dd51e30 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:23.007668 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f5b798-7a89-4f33-9d5e-5d8895841d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:23.008281 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5b798-7a89-4f33-9d5e-5d8895841d31 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:23.008560 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5b798-7a89-4f33-9d5e-5d8895841d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:23.008868 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5b798-7a89-4f33-9d5e-5d8895841d31 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:23.009518 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f5b798-7a89-4f33-9d5e-5d8895841d31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:23.011144 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f5b798-7a89-4f33-9d5e-5d8895841d31 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:23.022652 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1063/2104] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:50:23.027017 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a48b742-2815-4059-9317-c4d465635baf admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:23.037877 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a48b742-2815-4059-9317-c4d465635baf admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:23.042471 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5260b2-134d-498b-8b09-fa9ecb57356a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:23.043107 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5260b2-134d-498b-8b09-fa9ecb57356a admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:23.043389 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5260b2-134d-498b-8b09-fa9ecb57356a admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:23.043664 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5260b2-134d-498b-8b09-fa9ecb57356a admin admin] PATH_INFO: `/v3/projects/d7c6575f3b974d0d9dce503d749989f6/users/738ba32adfc9474f837b20e2a0afc319/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:23.059129 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5260b2-134d-498b-8b09-fa9ecb57356a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7c6575f3b974d0d9dce503d749989f6, user_id=738ba32adfc9474f837b20e2a0afc319, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:23.061844 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5260b2-134d-498b-8b09-fa9ecb57356a admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:23.138153 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1042/2105] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:50:23 2022] PUT /identity/v3/projects/d7c6575f3b974d0d9dce503d749989f6/users/738ba32adfc9474f837b20e2a0afc319/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:23.148795 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731ec49-7b59-4fbe-8016-5767b6f02089 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:23.149082 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731ec49-7b59-4fbe-8016-5767b6f02089 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:23.149358 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731ec49-7b59-4fbe-8016-5767b6f02089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:23.192691 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-5731ec49-7b59-4fbe-8016-5767b6f02089 None None] MFA Rules not processed for user `738ba32adfc9474f837b20e2a0afc319`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:23.194142 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-5731ec49-7b59-4fbe-8016-5767b6f02089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:23.261540 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1064/2106] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:50:23.266206 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5260b2-134d-498b-8b09-fa9ecb57356a admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:23.330106 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5260b2-134d-498b-8b09-fa9ecb57356a admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:23.335967 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291a0fc1-8c9c-43de-b9fa-c488931d2d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:23.336611 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291a0fc1-8c9c-43de-b9fa-c488931d2d49 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:23.337029 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291a0fc1-8c9c-43de-b9fa-c488931d2d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:23.337453 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291a0fc1-8c9c-43de-b9fa-c488931d2d49 admin admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:23.338089 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291a0fc1-8c9c-43de-b9fa-c488931d2d49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:23.340094 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291a0fc1-8c9c-43de-b9fa-c488931d2d49 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:23.445460 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1043/2107] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:50:23 2022] POST /identity/v3/projects => generated 375 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:50:23.450331 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5731ec49-7b59-4fbe-8016-5767b6f02089 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:23.463592 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5731ec49-7b59-4fbe-8016-5767b6f02089 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:23.522437 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb30d21f-8198-494c-908a-bfbdc0704523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:23.523315 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30d21f-8198-494c-908a-bfbdc0704523 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:23.523593 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30d21f-8198-494c-908a-bfbdc0704523 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:23.523864 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30d21f-8198-494c-908a-bfbdc0704523 admin admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:23.524563 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb30d21f-8198-494c-908a-bfbdc0704523 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:23.526568 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb30d21f-8198-494c-908a-bfbdc0704523 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:23.630919 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1065/2108] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:50:23 2022] POST /identity/v3/users => generated 294 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:50:23.637206 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291a0fc1-8c9c-43de-b9fa-c488931d2d49 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:23.648971 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291a0fc1-8c9c-43de-b9fa-c488931d2d49 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:23.653964 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda4e74-1bbd-4b58-bf23-ad97a27e3d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:23.654704 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda4e74-1bbd-4b58-bf23-ad97a27e3d4e admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:23.655089 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda4e74-1bbd-4b58-bf23-ad97a27e3d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:23.655462 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda4e74-1bbd-4b58-bf23-ad97a27e3d4e admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:23.656318 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeda4e74-1bbd-4b58-bf23-ad97a27e3d4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:23.658053 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeda4e74-1bbd-4b58-bf23-ad97a27e3d4e admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:23.672592 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1044/2109] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:50:23.675507 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb30d21f-8198-494c-908a-bfbdc0704523 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:23.687864 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb30d21f-8198-494c-908a-bfbdc0704523 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:23.698001 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31204ce4-5c59-4608-a59c-7a2b134b09de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:23.698938 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31204ce4-5c59-4608-a59c-7a2b134b09de admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:23.699256 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31204ce4-5c59-4608-a59c-7a2b134b09de admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:23.699678 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31204ce4-5c59-4608-a59c-7a2b134b09de admin admin] PATH_INFO: `/v3/projects/146797115c164cac8b9f270c2c3f6aa1/users/149fd97489c546288568873a3ea0c800/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:23.725077 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31204ce4-5c59-4608-a59c-7a2b134b09de admin admin] RBAC: Authorizing `identity:create_grant(project_id=146797115c164cac8b9f270c2c3f6aa1, user_id=149fd97489c546288568873a3ea0c800, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:23.725077 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31204ce4-5c59-4608-a59c-7a2b134b09de admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:23.792809 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1066/2110] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:50:23 2022] PUT /identity/v3/projects/146797115c164cac8b9f270c2c3f6aa1/users/149fd97489c546288568873a3ea0c800/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:50:23.796788 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda4e74-1bbd-4b58-bf23-ad97a27e3d4e admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:23.808060 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda4e74-1bbd-4b58-bf23-ad97a27e3d4e admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:23.843457 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:23.883614 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047052d3-419a-4359-be79-838e6a2afd51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:23.888005 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047052d3-419a-4359-be79-838e6a2afd51 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:23.888005 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047052d3-419a-4359-be79-838e6a2afd51 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:23.888005 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047052d3-419a-4359-be79-838e6a2afd51 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:23.888249 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31204ce4-5c59-4608-a59c-7a2b134b09de admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:23.888356 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047052d3-419a-4359-be79-838e6a2afd51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:23.888356 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047052d3-419a-4359-be79-838e6a2afd51 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:23.888498 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-31204ce4-5c59-4608-a59c-7a2b134b09de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:23.902140 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31204ce4-5c59-4608-a59c-7a2b134b09de admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:23.907218 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1c2bd0-04e7-4c54-959f-bdea9234995c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:23.907918 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1c2bd0-04e7-4c54-959f-bdea9234995c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:23.908274 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1c2bd0-04e7-4c54-959f-bdea9234995c admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:23.908803 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1c2bd0-04e7-4c54-959f-bdea9234995c admin admin] PATH_INFO: `/v3/users/48c1580f32f049eeb37a8b5fdc38d566` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:23.910130 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1c2bd0-04e7-4c54-959f-bdea9234995c admin admin] RBAC: Authorizing `identity:delete_user(user_id=48c1580f32f049eeb37a8b5fdc38d566)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:23.910524 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1045/2111] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:23 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:50:23.912222 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1c2bd0-04e7-4c54-959f-bdea9234995c admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:23.914348 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047052d3-419a-4359-be79-838e6a2afd51 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:23.928239 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047052d3-419a-4359-be79-838e6a2afd51 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:23.935490 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d261eb7-904d-4aaa-958a-9dff536e843b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:23.936378 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d261eb7-904d-4aaa-958a-9dff536e843b admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:23.936753 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d261eb7-904d-4aaa-958a-9dff536e843b admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:23.937141 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d261eb7-904d-4aaa-958a-9dff536e843b admin admin] PATH_INFO: `/v3/projects/146797115c164cac8b9f270c2c3f6aa1/users/149fd97489c546288568873a3ea0c800/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:23.939278 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d261eb7-904d-4aaa-958a-9dff536e843b admin admin] RBAC: Authorizing `identity:create_grant(project_id=146797115c164cac8b9f270c2c3f6aa1, user_id=149fd97489c546288568873a3ea0c800, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:23.943209 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d261eb7-904d-4aaa-958a-9dff536e843b admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:23.954433 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1046/2112] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:50:23 2022] PUT /identity/v3/projects/146797115c164cac8b9f270c2c3f6aa1/users/149fd97489c546288568873a3ea0c800/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:23.966750 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a0bf86-72f7-4b96-a13f-c6c27d7b39d5 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:23.966961 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a0bf86-72f7-4b96-a13f-c6c27d7b39d5 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:23.967240 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a0bf86-72f7-4b96-a13f-c6c27d7b39d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:23.996871 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-71a0bf86-72f7-4b96-a13f-c6c27d7b39d5 None None] MFA Rules not processed for user `149fd97489c546288568873a3ea0c800`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:23.998135 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-71a0bf86-72f7-4b96-a13f-c6c27d7b39d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:24.079285 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:INSERT=5,keystone:SELECT=90,keystone:DELETE=5 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:24.087317 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1047/2113] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:23 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:50:24.100240 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-6b1c2bd0-04e7-4c54-959f-bdea9234995c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48c1580f32f049eeb37a8b5fdc38d566'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:24.106352 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a0bf86-72f7-4b96-a13f-c6c27d7b39d5 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:24.106352 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-71a0bf86-72f7-4b96-a13f-c6c27d7b39d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:24.177509 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a0bf86-72f7-4b96-a13f-c6c27d7b39d5 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:24.181291 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e04d1-670b-452a-a82a-61a864401119 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:24.181885 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e04d1-670b-452a-a82a-61a864401119 service cinder] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:24.182161 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e04d1-670b-452a-a82a-61a864401119 service cinder] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:24.182430 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e04d1-670b-452a-a82a-61a864401119 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:24.186151 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-6b1c2bd0-04e7-4c54-959f-bdea9234995c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48c1580f32f049eeb37a8b5fdc38d566'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:24.198800 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-6b1c2bd0-04e7-4c54-959f-bdea9234995c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48c1580f32f049eeb37a8b5fdc38d566'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:24.206472 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-6b1c2bd0-04e7-4c54-959f-bdea9234995c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48c1580f32f049eeb37a8b5fdc38d566'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:24.207840 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-6b1c2bd0-04e7-4c54-959f-bdea9234995c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48c1580f32f049eeb37a8b5fdc38d566'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:24.225108 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1067/2114] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:50:23 2022] DELETE /identity/v3/users/48c1580f32f049eeb37a8b5fdc38d566 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:24.225108 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54abd811-d7d0-42c8-b80f-47c2bedaaeee None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:24.225108 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54abd811-d7d0-42c8-b80f-47c2bedaaeee None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:24.225108 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54abd811-d7d0-42c8-b80f-47c2bedaaeee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:24.244982 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-54abd811-d7d0-42c8-b80f-47c2bedaaeee None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:24.246245 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-54abd811-d7d0-42c8-b80f-47c2bedaaeee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:24.249595 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:24.261792 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e04d1-670b-452a-a82a-61a864401119 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:24.261792 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e04d1-670b-452a-a82a-61a864401119 service cinder] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:24.277482 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1048/2115] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:50:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:50:24.329115 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1068/2116] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:50:24.341125 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e04d1-670b-452a-a82a-61a864401119 service cinder] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:24.341616 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-991e04d1-670b-452a-a82a-61a864401119 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:24.429296 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e04d1-670b-452a-a82a-61a864401119 service cinder] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:24.484535 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc918d15-0364-4ef1-a50e-19f8bb461bad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:24.485131 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc918d15-0364-4ef1-a50e-19f8bb461bad service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:24.485404 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc918d15-0364-4ef1-a50e-19f8bb461bad service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:24.485681 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc918d15-0364-4ef1-a50e-19f8bb461bad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:24.496885 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc918d15-0364-4ef1-a50e-19f8bb461bad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:24.500073 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc918d15-0364-4ef1-a50e-19f8bb461bad service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:24.516895 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1049/2117] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:50:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:50:25.045798 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54abd811-d7d0-42c8-b80f-47c2bedaaeee None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:25.048188 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-54abd811-d7d0-42c8-b80f-47c2bedaaeee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:25.094262 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54abd811-d7d0-42c8-b80f-47c2bedaaeee None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:25.107111 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55087839-b793-4af8-89b0-19820cf5fc01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:25.114357 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55087839-b793-4af8-89b0-19820cf5fc01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:25.114662 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55087839-b793-4af8-89b0-19820cf5fc01 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:25.114939 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55087839-b793-4af8-89b0-19820cf5fc01 admin admin] PATH_INFO: `/v3/projects/f5d16a389a22469d983ca8fd2df60644` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:25.118329 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55087839-b793-4af8-89b0-19820cf5fc01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5d16a389a22469d983ca8fd2df60644)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:25.120340 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55087839-b793-4af8-89b0-19820cf5fc01 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:25.276411 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-55087839-b793-4af8-89b0-19820cf5fc01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5d16a389a22469d983ca8fd2df60644', 'request_id': 'req-55087839-b793-4af8-89b0-19820cf5fc01'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:25.282897 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-55087839-b793-4af8-89b0-19820cf5fc01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5d16a389a22469d983ca8fd2df60644', 'request_id': 'req-55087839-b793-4af8-89b0-19820cf5fc01'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:25.290267 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-55087839-b793-4af8-89b0-19820cf5fc01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5d16a389a22469d983ca8fd2df60644', 'request_id': 'req-55087839-b793-4af8-89b0-19820cf5fc01'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:25.403153 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-55087839-b793-4af8-89b0-19820cf5fc01 admin admin] The token cache is being invalidate because project f5d16a389a22469d983ca8fd2df60644 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:50:25.403529 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-55087839-b793-4af8-89b0-19820cf5fc01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:25.404296 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-55087839-b793-4af8-89b0-19820cf5fc01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:25.405915 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1069/2118] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:50:25 2022] DELETE /identity/v3/projects/f5d16a389a22469d983ca8fd2df60644 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:25.410609 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc918d15-0364-4ef1-a50e-19f8bb461bad service neutron] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:25.420249 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc918d15-0364-4ef1-a50e-19f8bb461bad service neutron] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:25.491948 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ad4fe-56c8-4f5c-a403-5cf910b65bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:25.492675 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ad4fe-56c8-4f5c-a403-5cf910b65bb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:25.492994 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ad4fe-56c8-4f5c-a403-5cf910b65bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:25.493271 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ad4fe-56c8-4f5c-a403-5cf910b65bb7 admin admin] PATH_INFO: `/v3/users/df4177fc0f844dfd9d32bbaf175b4004` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:25.494407 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ad4fe-56c8-4f5c-a403-5cf910b65bb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df4177fc0f844dfd9d32bbaf175b4004)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:25.496350 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ad4fe-56c8-4f5c-a403-5cf910b65bb7 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:25.752681 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-955ad4fe-56c8-4f5c-a403-5cf910b65bb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df4177fc0f844dfd9d32bbaf175b4004'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:25.755318 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:25.766802 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-955ad4fe-56c8-4f5c-a403-5cf910b65bb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df4177fc0f844dfd9d32bbaf175b4004'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:25.778292 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-955ad4fe-56c8-4f5c-a403-5cf910b65bb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df4177fc0f844dfd9d32bbaf175b4004'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:25.868522 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-955ad4fe-56c8-4f5c-a403-5cf910b65bb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df4177fc0f844dfd9d32bbaf175b4004'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:25.870967 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-955ad4fe-56c8-4f5c-a403-5cf910b65bb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df4177fc0f844dfd9d32bbaf175b4004'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:25.891688 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1050/2119] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:50:25 2022] DELETE /identity/v3/users/df4177fc0f844dfd9d32bbaf175b4004 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:26.355534 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55087839-b793-4af8-89b0-19820cf5fc01 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:26.357026 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-55087839-b793-4af8-89b0-19820cf5fc01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:26.390482 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55087839-b793-4af8-89b0-19820cf5fc01 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:26.450396 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:26.483161 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9859cf-301a-45c2-ab9b-e9c88be37555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:26.483730 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9859cf-301a-45c2-ab9b-e9c88be37555 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:26.484078 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9859cf-301a-45c2-ab9b-e9c88be37555 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:26.484472 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9859cf-301a-45c2-ab9b-e9c88be37555 admin admin] PATH_INFO: `/v3/projects/5c289e6fa3ff40feaf37d07155e7da23` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:26.486478 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9859cf-301a-45c2-ab9b-e9c88be37555 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c289e6fa3ff40feaf37d07155e7da23)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:26.489208 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9859cf-301a-45c2-ab9b-e9c88be37555 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:26.747654 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-6e9859cf-301a-45c2-ab9b-e9c88be37555 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c289e6fa3ff40feaf37d07155e7da23', 'request_id': 'req-6e9859cf-301a-45c2-ab9b-e9c88be37555'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:26.753267 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-6e9859cf-301a-45c2-ab9b-e9c88be37555 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c289e6fa3ff40feaf37d07155e7da23', 'request_id': 'req-6e9859cf-301a-45c2-ab9b-e9c88be37555'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:26.760692 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-6e9859cf-301a-45c2-ab9b-e9c88be37555 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c289e6fa3ff40feaf37d07155e7da23', 'request_id': 'req-6e9859cf-301a-45c2-ab9b-e9c88be37555'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:26.883257 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-6e9859cf-301a-45c2-ab9b-e9c88be37555 admin admin] The token cache is being invalidate because project 5c289e6fa3ff40feaf37d07155e7da23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:50:26.883471 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-6e9859cf-301a-45c2-ab9b-e9c88be37555 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:26.884092 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-6e9859cf-301a-45c2-ab9b-e9c88be37555 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:26.885633 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1070/2120] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:50:26 2022] DELETE /identity/v3/projects/5c289e6fa3ff40feaf37d07155e7da23 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:26.889716 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ad4fe-56c8-4f5c-a403-5cf910b65bb7 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:26.892935 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-955ad4fe-56c8-4f5c-a403-5cf910b65bb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:26.904159 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ad4fe-56c8-4f5c-a403-5cf910b65bb7 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:26.977500 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1debea5f-e5f2-4f96-8f82-01895e5c1737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:26.978213 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1debea5f-e5f2-4f96-8f82-01895e5c1737 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:26.978627 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1debea5f-e5f2-4f96-8f82-01895e5c1737 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:26.978801 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1debea5f-e5f2-4f96-8f82-01895e5c1737 admin admin] PATH_INFO: `/v3/users/7d02f460b0e745b09813fe405c0cb400` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:26.979941 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1debea5f-e5f2-4f96-8f82-01895e5c1737 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d02f460b0e745b09813fe405c0cb400)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:26.981950 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1debea5f-e5f2-4f96-8f82-01895e5c1737 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:27.226849 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-1debea5f-e5f2-4f96-8f82-01895e5c1737 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d02f460b0e745b09813fe405c0cb400'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:27.249419 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-1debea5f-e5f2-4f96-8f82-01895e5c1737 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d02f460b0e745b09813fe405c0cb400'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:27.261999 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-1debea5f-e5f2-4f96-8f82-01895e5c1737 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d02f460b0e745b09813fe405c0cb400'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:27.378292 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-1debea5f-e5f2-4f96-8f82-01895e5c1737 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d02f460b0e745b09813fe405c0cb400'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:27.379640 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-1debea5f-e5f2-4f96-8f82-01895e5c1737 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d02f460b0e745b09813fe405c0cb400'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:27.387022 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1051/2121] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:50:26 2022] DELETE /identity/v3/users/7d02f460b0e745b09813fe405c0cb400 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:27.678334 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9859cf-301a-45c2-ab9b-e9c88be37555 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:27.679805 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-6e9859cf-301a-45c2-ab9b-e9c88be37555 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:27.706226 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9859cf-301a-45c2-ab9b-e9c88be37555 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:27.791642 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc0c15-4736-4064-ab85-2905ac60bd34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:27.792287 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc0c15-4736-4064-ab85-2905ac60bd34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:27.792583 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc0c15-4736-4064-ab85-2905ac60bd34 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:27.792884 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc0c15-4736-4064-ab85-2905ac60bd34 admin admin] PATH_INFO: `/v3/users/738ba32adfc9474f837b20e2a0afc319` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:27.794252 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cc0c15-4736-4064-ab85-2905ac60bd34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=738ba32adfc9474f837b20e2a0afc319)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:27.796230 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cc0c15-4736-4064-ab85-2905ac60bd34 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:28.032804 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-72cc0c15-4736-4064-ab85-2905ac60bd34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '738ba32adfc9474f837b20e2a0afc319'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:28.121276 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-72cc0c15-4736-4064-ab85-2905ac60bd34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '738ba32adfc9474f837b20e2a0afc319'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:28.131523 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:28.137190 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-72cc0c15-4736-4064-ab85-2905ac60bd34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '738ba32adfc9474f837b20e2a0afc319'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:28.144786 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-72cc0c15-4736-4064-ab85-2905ac60bd34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '738ba32adfc9474f837b20e2a0afc319'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:28.145523 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-72cc0c15-4736-4064-ab85-2905ac60bd34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '738ba32adfc9474f837b20e2a0afc319'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:28.151440 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1071/2122] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:50:27 2022] DELETE /identity/v3/users/738ba32adfc9474f837b20e2a0afc319 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:28.156242 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa248f-8fdb-43af-ad11-ac04bbc11a92 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:28.156732 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa248f-8fdb-43af-ad11-ac04bbc11a92 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:28.156854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa248f-8fdb-43af-ad11-ac04bbc11a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:28.180765 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-5daa248f-8fdb-43af-ad11-ac04bbc11a92 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:28.181034 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-5daa248f-8fdb-43af-ad11-ac04bbc11a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:28.227776 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:28.247051 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1052/2123] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:50:28.256557 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc0c15-4736-4064-ab85-2905ac60bd34 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:28.258014 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-72cc0c15-4736-4064-ab85-2905ac60bd34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:28.306177 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daa248f-8fdb-43af-ad11-ac04bbc11a92 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:28.308280 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-5daa248f-8fdb-43af-ad11-ac04bbc11a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:28.319460 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc0c15-4736-4064-ab85-2905ac60bd34 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:28.324790 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e8859e-f03b-448d-a8ad-8617b856ca4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:28.325395 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e8859e-f03b-448d-a8ad-8617b856ca4d service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:28.325772 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e8859e-f03b-448d-a8ad-8617b856ca4d service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:28.326166 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e8859e-f03b-448d-a8ad-8617b856ca4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:28.327134 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daa248f-8fdb-43af-ad11-ac04bbc11a92 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:28.334648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffb557a-3bc7-4eb4-ba1b-f2dcc9356127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:28.337355 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffb557a-3bc7-4eb4-ba1b-f2dcc9356127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:28.338021 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffb557a-3bc7-4eb4-ba1b-f2dcc9356127 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:28.338176 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffb557a-3bc7-4eb4-ba1b-f2dcc9356127 admin admin] PATH_INFO: `/v3/projects/0c83a8bd08254a3181e3a6f3768b81f8` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:28.340887 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e8859e-f03b-448d-a8ad-8617b856ca4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:28.340990 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffb557a-3bc7-4eb4-ba1b-f2dcc9356127 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c83a8bd08254a3181e3a6f3768b81f8)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:28.345757 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e8859e-f03b-448d-a8ad-8617b856ca4d service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:28.345855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffb557a-3bc7-4eb4-ba1b-f2dcc9356127 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:28.355526 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1072/2124] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:50:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:50:28.451339 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-2ffb557a-3bc7-4eb4-ba1b-f2dcc9356127 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c83a8bd08254a3181e3a6f3768b81f8', 'request_id': 'req-2ffb557a-3bc7-4eb4-ba1b-f2dcc9356127'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:28.563573 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-2ffb557a-3bc7-4eb4-ba1b-f2dcc9356127 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c83a8bd08254a3181e3a6f3768b81f8', 'request_id': 'req-2ffb557a-3bc7-4eb4-ba1b-f2dcc9356127'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:28.570597 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-2ffb557a-3bc7-4eb4-ba1b-f2dcc9356127 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c83a8bd08254a3181e3a6f3768b81f8', 'request_id': 'req-2ffb557a-3bc7-4eb4-ba1b-f2dcc9356127'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:28.582426 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-2ffb557a-3bc7-4eb4-ba1b-f2dcc9356127 admin admin] The token cache is being invalidate because project 0c83a8bd08254a3181e3a6f3768b81f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:50:28.582680 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-2ffb557a-3bc7-4eb4-ba1b-f2dcc9356127 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:28.583348 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-2ffb557a-3bc7-4eb4-ba1b-f2dcc9356127 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:28.585351 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1053/2125] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:50:28 2022] DELETE /identity/v3/projects/0c83a8bd08254a3181e3a6f3768b81f8 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:28.590554 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891ad39-355e-45fd-9e86-33d1a6538ce5 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:28.595290 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891ad39-355e-45fd-9e86-33d1a6538ce5 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:28.595290 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891ad39-355e-45fd-9e86-33d1a6538ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:28.611668 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-b891ad39-355e-45fd-9e86-33d1a6538ce5 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:28.613806 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-b891ad39-355e-45fd-9e86-33d1a6538ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:28.705625 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1073/2126] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:50:28.714679 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88543d96-9484-43f1-973c-8fbf6e184499 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:28.715211 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88543d96-9484-43f1-973c-8fbf6e184499 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:28.716467 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88543d96-9484-43f1-973c-8fbf6e184499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:28.755845 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-88543d96-9484-43f1-973c-8fbf6e184499 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:28.757526 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-88543d96-9484-43f1-973c-8fbf6e184499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:28.788184 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1054/2127] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:50:28.790527 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b891ad39-355e-45fd-9e86-33d1a6538ce5 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:28.802002 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b891ad39-355e-45fd-9e86-33d1a6538ce5 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:28.809856 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed2890a-68b5-45c5-94ce-3b594469eea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:28.810812 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed2890a-68b5-45c5-94ce-3b594469eea9 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:28.811243 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed2890a-68b5-45c5-94ce-3b594469eea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:28.811658 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed2890a-68b5-45c5-94ce-3b594469eea9 admin admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:28.812478 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed2890a-68b5-45c5-94ce-3b594469eea9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:50:28.812836 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed2890a-68b5-45c5-94ce-3b594469eea9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:28.815136 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed2890a-68b5-45c5-94ce-3b594469eea9 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:28.839591 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1074/2128] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:50:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:50:28.842431 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88543d96-9484-43f1-973c-8fbf6e184499 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:28.853236 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88543d96-9484-43f1-973c-8fbf6e184499 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:28.858841 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe66ea-a8ef-41bf-ad23-257ca2461f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:28.859561 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe66ea-a8ef-41bf-ad23-257ca2461f34 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:28.859836 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe66ea-a8ef-41bf-ad23-257ca2461f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:28.859982 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe66ea-a8ef-41bf-ad23-257ca2461f34 admin admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:28.860759 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbe66ea-a8ef-41bf-ad23-257ca2461f34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:28.872022 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbe66ea-a8ef-41bf-ad23-257ca2461f34 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:28.985065 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1055/2129] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:50:28 2022] POST /identity/v3/projects => generated 371 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:50:28.991620 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed2890a-68b5-45c5-94ce-3b594469eea9 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:29.004485 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed2890a-68b5-45c5-94ce-3b594469eea9 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:29.031874 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe66ea-a8ef-41bf-ad23-257ca2461f34 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:29.031874 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-9dbe66ea-a8ef-41bf-ad23-257ca2461f34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:29.117188 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91342d22-c780-49dc-83f6-d0523f2fbea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:29.117188 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91342d22-c780-49dc-83f6-d0523f2fbea7 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:29.117400 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91342d22-c780-49dc-83f6-d0523f2fbea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:29.117684 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91342d22-c780-49dc-83f6-d0523f2fbea7 admin admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:29.118326 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91342d22-c780-49dc-83f6-d0523f2fbea7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:29.120625 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91342d22-c780-49dc-83f6-d0523f2fbea7 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:29.149654 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe66ea-a8ef-41bf-ad23-257ca2461f34 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:29.154485 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f334b-7ac6-495f-af46-08224f8bf375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:29.155115 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f334b-7ac6-495f-af46-08224f8bf375 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:29.155396 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f334b-7ac6-495f-af46-08224f8bf375 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:29.155671 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f334b-7ac6-495f-af46-08224f8bf375 admin admin] PATH_INFO: `/v3/projects/d7c6575f3b974d0d9dce503d749989f6` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:29.156985 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1f334b-7ac6-495f-af46-08224f8bf375 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7c6575f3b974d0d9dce503d749989f6)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:29.159255 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1f334b-7ac6-495f-af46-08224f8bf375 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:29.241314 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1075/2130] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:50:28 2022] POST /identity/v3/users => generated 286 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:50:29.242312 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91342d22-c780-49dc-83f6-d0523f2fbea7 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:29.250394 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:29.256154 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91342d22-c780-49dc-83f6-d0523f2fbea7 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:29.262080 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d425022-adfd-489c-befb-4ab85b368344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:29.262669 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d425022-adfd-489c-befb-4ab85b368344 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:29.266650 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d425022-adfd-489c-befb-4ab85b368344 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:29.266650 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d425022-adfd-489c-befb-4ab85b368344 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:29.266650 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d425022-adfd-489c-befb-4ab85b368344 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:29.266650 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d425022-adfd-489c-befb-4ab85b368344 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:29.279208 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1076/2131] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:50:29.284397 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d425022-adfd-489c-befb-4ab85b368344 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:29.293124 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:29.297173 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-2a1f334b-7ac6-495f-af46-08224f8bf375 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7c6575f3b974d0d9dce503d749989f6', 'request_id': 'req-2a1f334b-7ac6-495f-af46-08224f8bf375'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:29.301988 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d425022-adfd-489c-befb-4ab85b368344 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:29.357091 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2202a6b2-c13d-4272-a709-0698aa8e1cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:29.357697 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202a6b2-c13d-4272-a709-0698aa8e1cc1 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:29.357981 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202a6b2-c13d-4272-a709-0698aa8e1cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:29.358282 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202a6b2-c13d-4272-a709-0698aa8e1cc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:29.358929 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2202a6b2-c13d-4272-a709-0698aa8e1cc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:29.360527 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2202a6b2-c13d-4272-a709-0698aa8e1cc1 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:29.366536 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-2a1f334b-7ac6-495f-af46-08224f8bf375 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7c6575f3b974d0d9dce503d749989f6', 'request_id': 'req-2a1f334b-7ac6-495f-af46-08224f8bf375'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:29.374585 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1077/2132] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:50:29.374743 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-2a1f334b-7ac6-495f-af46-08224f8bf375 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7c6575f3b974d0d9dce503d749989f6', 'request_id': 'req-2a1f334b-7ac6-495f-af46-08224f8bf375'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:29.375960 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2202a6b2-c13d-4272-a709-0698aa8e1cc1 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:29.385941 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-2a1f334b-7ac6-495f-af46-08224f8bf375 admin admin] The token cache is being invalidate because project d7c6575f3b974d0d9dce503d749989f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:50:29.386230 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-2a1f334b-7ac6-495f-af46-08224f8bf375 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:29.386898 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-2a1f334b-7ac6-495f-af46-08224f8bf375 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:29.387844 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2202a6b2-c13d-4272-a709-0698aa8e1cc1 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:29.388497 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1056/2133] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:50:29 2022] DELETE /identity/v3/projects/d7c6575f3b974d0d9dce503d749989f6 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:29.392292 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f334b-7ac6-495f-af46-08224f8bf375 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:29.395526 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-2a1f334b-7ac6-495f-af46-08224f8bf375 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:29.396666 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1fed43-3383-4e9d-bbd4-fbad9a026114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:29.397435 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1fed43-3383-4e9d-bbd4-fbad9a026114 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:29.399369 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1fed43-3383-4e9d-bbd4-fbad9a026114 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:29.399369 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1fed43-3383-4e9d-bbd4-fbad9a026114 admin admin] PATH_INFO: `/v3/projects/4f38e29d63fd4d04984af578b1708e4a/users/9c3c51d98e274150933a1306f7747f42/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:29.408824 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f334b-7ac6-495f-af46-08224f8bf375 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:29.414527 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1fed43-3383-4e9d-bbd4-fbad9a026114 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f38e29d63fd4d04984af578b1708e4a, user_id=9c3c51d98e274150933a1306f7747f42, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:29.415581 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633875f5-5752-489d-8c18-261a8dd80d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:29.416224 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633875f5-5752-489d-8c18-261a8dd80d1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:29.416786 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633875f5-5752-489d-8c18-261a8dd80d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:29.417282 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633875f5-5752-489d-8c18-261a8dd80d1e admin admin] PATH_INFO: `/v3/users/149fd97489c546288568873a3ea0c800` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:29.418537 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633875f5-5752-489d-8c18-261a8dd80d1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=149fd97489c546288568873a3ea0c800)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:29.420695 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633875f5-5752-489d-8c18-261a8dd80d1e admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:29.420820 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1fed43-3383-4e9d-bbd4-fbad9a026114 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:29.496024 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1078/2134] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:50:29 2022] PUT /identity/v3/projects/4f38e29d63fd4d04984af578b1708e4a/users/9c3c51d98e274150933a1306f7747f42/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:50:29.496987 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1fed43-3383-4e9d-bbd4-fbad9a026114 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:29.498643 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-ca1fed43-3383-4e9d-bbd4-fbad9a026114 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:29.565088 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1fed43-3383-4e9d-bbd4-fbad9a026114 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:29.570572 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91436598-9311-42e8-913d-2aef7da05950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:29.571241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91436598-9311-42e8-913d-2aef7da05950 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:29.576099 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91436598-9311-42e8-913d-2aef7da05950 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:29.576099 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91436598-9311-42e8-913d-2aef7da05950 admin admin] PATH_INFO: `/v3/users/0a23eaf72e834ccebbec25dbddf4964e` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:29.576099 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91436598-9311-42e8-913d-2aef7da05950 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a23eaf72e834ccebbec25dbddf4964e)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:29.576099 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91436598-9311-42e8-913d-2aef7da05950 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:29.742973 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-633875f5-5752-489d-8c18-261a8dd80d1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '149fd97489c546288568873a3ea0c800'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:29.754767 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-633875f5-5752-489d-8c18-261a8dd80d1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '149fd97489c546288568873a3ea0c800'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:29.761734 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-633875f5-5752-489d-8c18-261a8dd80d1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '149fd97489c546288568873a3ea0c800'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:29.955681 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-633875f5-5752-489d-8c18-261a8dd80d1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '149fd97489c546288568873a3ea0c800'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:29.956411 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-633875f5-5752-489d-8c18-261a8dd80d1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '149fd97489c546288568873a3ea0c800'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:29.964272 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1057/2135] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:50:29 2022] DELETE /identity/v3/users/149fd97489c546288568873a3ea0c800 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:29.965739 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a6b00d-9ba1-4f69-8a4d-c1db3ac7ceef None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:29.966202 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a6b00d-9ba1-4f69-8a4d-c1db3ac7ceef None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:29.966523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a6b00d-9ba1-4f69-8a4d-c1db3ac7ceef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:29.986776 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-91436598-9311-42e8-913d-2aef7da05950 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a23eaf72e834ccebbec25dbddf4964e'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:29.989958 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-00a6b00d-9ba1-4f69-8a4d-c1db3ac7ceef None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:29.991441 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-00a6b00d-9ba1-4f69-8a4d-c1db3ac7ceef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:30.064237 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1058/2136] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:50:30.066160 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-91436598-9311-42e8-913d-2aef7da05950 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a23eaf72e834ccebbec25dbddf4964e'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:30.073926 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a6b00d-9ba1-4f69-8a4d-c1db3ac7ceef None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:30.075298 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-00a6b00d-9ba1-4f69-8a4d-c1db3ac7ceef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:30.080010 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-91436598-9311-42e8-913d-2aef7da05950 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a23eaf72e834ccebbec25dbddf4964e'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:30.088674 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-91436598-9311-42e8-913d-2aef7da05950 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a23eaf72e834ccebbec25dbddf4964e'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:30.089436 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-91436598-9311-42e8-913d-2aef7da05950 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a23eaf72e834ccebbec25dbddf4964e'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:30.095438 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1079/2137] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:50:29 2022] DELETE /identity/v3/users/0a23eaf72e834ccebbec25dbddf4964e => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:30.100490 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59baa85f-a91b-4b75-8efc-96467cf49520 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:30.102303 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59baa85f-a91b-4b75-8efc-96467cf49520 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:30.102602 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59baa85f-a91b-4b75-8efc-96467cf49520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:30.123582 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-59baa85f-a91b-4b75-8efc-96467cf49520 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:30.125309 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-59baa85f-a91b-4b75-8efc-96467cf49520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:30.140429 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a6b00d-9ba1-4f69-8a4d-c1db3ac7ceef None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:30.173477 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:30.212834 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda9dac4-2662-4b0e-a5b2-a534e3b36dba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:30.213385 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9dac4-2662-4b0e-a5b2-a534e3b36dba service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:30.213652 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9dac4-2662-4b0e-a5b2-a534e3b36dba service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:30.213928 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9dac4-2662-4b0e-a5b2-a534e3b36dba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:30.215726 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-eda9dac4-2662-4b0e-a5b2-a534e3b36dba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:30.217928 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1080/2138] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:50:30.225127 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:30.232071 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59baa85f-a91b-4b75-8efc-96467cf49520 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:30.233826 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-59baa85f-a91b-4b75-8efc-96467cf49520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:30.236551 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda9dac4-2662-4b0e-a5b2-a534e3b36dba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:30.238566 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda9dac4-2662-4b0e-a5b2-a534e3b36dba service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:30.247289 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59baa85f-a91b-4b75-8efc-96467cf49520 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:30.251396 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a5d18d-1c79-4b82-b1d7-9167a4f6aba6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:30.252221 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5d18d-1c79-4b82-b1d7-9167a4f6aba6 service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:30.252440 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5d18d-1c79-4b82-b1d7-9167a4f6aba6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:30.252751 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5d18d-1c79-4b82-b1d7-9167a4f6aba6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:30.257410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1059/2139] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:50:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:50:30.277637 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a5d18d-1c79-4b82-b1d7-9167a4f6aba6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:30.279187 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a5d18d-1c79-4b82-b1d7-9167a4f6aba6 service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:30.291042 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1081/2140] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:50:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:50:30.620662 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda9dac4-2662-4b0e-a5b2-a534e3b36dba service neutron] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:30.620662 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-eda9dac4-2662-4b0e-a5b2-a534e3b36dba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:30.638525 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda9dac4-2662-4b0e-a5b2-a534e3b36dba service neutron] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:30.652218 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96a3090-6779-4ce1-84a1-b41492741ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:30.653265 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a3090-6779-4ce1-84a1-b41492741ea1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:30.653489 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a3090-6779-4ce1-84a1-b41492741ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:30.654589 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a3090-6779-4ce1-84a1-b41492741ea1 admin admin] PATH_INFO: `/v3/projects/146797115c164cac8b9f270c2c3f6aa1` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:30.656878 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96a3090-6779-4ce1-84a1-b41492741ea1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=146797115c164cac8b9f270c2c3f6aa1)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:30.659399 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96a3090-6779-4ce1-84a1-b41492741ea1 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:30.789154 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-c96a3090-6779-4ce1-84a1-b41492741ea1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '146797115c164cac8b9f270c2c3f6aa1', 'request_id': 'req-c96a3090-6779-4ce1-84a1-b41492741ea1'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:30.906858 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-c96a3090-6779-4ce1-84a1-b41492741ea1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '146797115c164cac8b9f270c2c3f6aa1', 'request_id': 'req-c96a3090-6779-4ce1-84a1-b41492741ea1'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:30.914506 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-c96a3090-6779-4ce1-84a1-b41492741ea1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '146797115c164cac8b9f270c2c3f6aa1', 'request_id': 'req-c96a3090-6779-4ce1-84a1-b41492741ea1'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:30.927817 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-c96a3090-6779-4ce1-84a1-b41492741ea1 admin admin] The token cache is being invalidate because project 146797115c164cac8b9f270c2c3f6aa1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:50:30.927988 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-c96a3090-6779-4ce1-84a1-b41492741ea1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:30.928958 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-c96a3090-6779-4ce1-84a1-b41492741ea1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:30.930640 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1060/2141] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:50:30 2022] DELETE /identity/v3/projects/146797115c164cac8b9f270c2c3f6aa1 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:30.935454 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c8c55-c048-4153-9157-cbe32e90fabf None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:30.936211 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c8c55-c048-4153-9157-cbe32e90fabf None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:30.936211 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c8c55-c048-4153-9157-cbe32e90fabf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:30.942014 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96a3090-6779-4ce1-84a1-b41492741ea1 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:30.945850 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-c96a3090-6779-4ce1-84a1-b41492741ea1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:30.966205 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-652c8c55-c048-4153-9157-cbe32e90fabf None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:30.969416 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-652c8c55-c048-4153-9157-cbe32e90fabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:31.038425 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96a3090-6779-4ce1-84a1-b41492741ea1 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:31.043758 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec065542-1b7b-4394-8e29-f7f887a7a72d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:31.044288 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec065542-1b7b-4394-8e29-f7f887a7a72d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:31.044568 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec065542-1b7b-4394-8e29-f7f887a7a72d admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:31.044927 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec065542-1b7b-4394-8e29-f7f887a7a72d admin admin] PATH_INFO: `/v3/projects/7dbca51cb31642a6a965e1d7171d5f54` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:31.046531 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec065542-1b7b-4394-8e29-f7f887a7a72d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dbca51cb31642a6a965e1d7171d5f54)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:31.047287 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1082/2142] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:50:31.048549 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec065542-1b7b-4394-8e29-f7f887a7a72d admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:31.060804 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d37fdf-082e-4961-b100-16d8fe9f736c None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:31.063468 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d37fdf-082e-4961-b100-16d8fe9f736c None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:31.064658 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d37fdf-082e-4961-b100-16d8fe9f736c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:31.106799 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-94d37fdf-082e-4961-b100-16d8fe9f736c None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:31.108012 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-94d37fdf-082e-4961-b100-16d8fe9f736c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:31.117443 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1083/2143] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:50:31.121993 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d37fdf-082e-4961-b100-16d8fe9f736c None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:31.132682 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d37fdf-082e-4961-b100-16d8fe9f736c None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:31.137686 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee0d778-29c6-4bae-8aea-36f0f84461d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:31.138292 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee0d778-29c6-4bae-8aea-36f0f84461d4 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:31.138573 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee0d778-29c6-4bae-8aea-36f0f84461d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:31.138845 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee0d778-29c6-4bae-8aea-36f0f84461d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:31.139527 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee0d778-29c6-4bae-8aea-36f0f84461d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:50:31.139826 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee0d778-29c6-4bae-8aea-36f0f84461d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:31.141597 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee0d778-29c6-4bae-8aea-36f0f84461d4 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:31.154972 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1084/2144] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:50:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:50:31.159200 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee0d778-29c6-4bae-8aea-36f0f84461d4 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:31.170417 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee0d778-29c6-4bae-8aea-36f0f84461d4 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:31.175668 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:31.193971 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-ec065542-1b7b-4394-8e29-f7f887a7a72d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dbca51cb31642a6a965e1d7171d5f54', 'request_id': 'req-ec065542-1b7b-4394-8e29-f7f887a7a72d'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:31.196686 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599958bf-f01f-43cc-bcf7-8c44d04cbeb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:31.198041 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599958bf-f01f-43cc-bcf7-8c44d04cbeb3 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:31.198506 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599958bf-f01f-43cc-bcf7-8c44d04cbeb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:31.198920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599958bf-f01f-43cc-bcf7-8c44d04cbeb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:31.200117 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599958bf-f01f-43cc-bcf7-8c44d04cbeb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:31.203216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599958bf-f01f-43cc-bcf7-8c44d04cbeb3 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:31.274878 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-ec065542-1b7b-4394-8e29-f7f887a7a72d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dbca51cb31642a6a965e1d7171d5f54', 'request_id': 'req-ec065542-1b7b-4394-8e29-f7f887a7a72d'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:31.282062 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-ec065542-1b7b-4394-8e29-f7f887a7a72d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dbca51cb31642a6a965e1d7171d5f54', 'request_id': 'req-ec065542-1b7b-4394-8e29-f7f887a7a72d'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:31.293799 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-ec065542-1b7b-4394-8e29-f7f887a7a72d admin admin] The token cache is being invalidate because project 7dbca51cb31642a6a965e1d7171d5f54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:50:31.294085 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-ec065542-1b7b-4394-8e29-f7f887a7a72d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:31.294794 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-ec065542-1b7b-4394-8e29-f7f887a7a72d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:31.296500 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1061/2145] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:50:30 2022] DELETE /identity/v3/projects/7dbca51cb31642a6a965e1d7171d5f54 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:31.301013 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec065542-1b7b-4394-8e29-f7f887a7a72d admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:31.301692 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-ec065542-1b7b-4394-8e29-f7f887a7a72d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:31.315680 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1085/2146] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:50:31 2022] POST /identity/v3/projects => generated 367 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:50:31.333317 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599958bf-f01f-43cc-bcf7-8c44d04cbeb3 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:31.336236 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-599958bf-f01f-43cc-bcf7-8c44d04cbeb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:31.381555 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:31.389257 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec065542-1b7b-4394-8e29-f7f887a7a72d admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:31.419470 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599958bf-f01f-43cc-bcf7-8c44d04cbeb3 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:31.424630 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0b06c9-1d1f-4c53-81f1-0f44daa98bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:31.426001 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0b06c9-1d1f-4c53-81f1-0f44daa98bdd admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:31.426287 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0b06c9-1d1f-4c53-81f1-0f44daa98bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:31.426812 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0b06c9-1d1f-4c53-81f1-0f44daa98bdd admin admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:31.427548 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0b06c9-1d1f-4c53-81f1-0f44daa98bdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:31.429895 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0b06c9-1d1f-4c53-81f1-0f44daa98bdd admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:31.469495 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e6d4db-09dd-4af6-bcc5-e391d84bc0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:31.470231 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6d4db-09dd-4af6-bcc5-e391d84bc0f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:31.471130 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6d4db-09dd-4af6-bcc5-e391d84bc0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:31.471484 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6d4db-09dd-4af6-bcc5-e391d84bc0f0 admin admin] PATH_INFO: `/v3/users/8ffffc6cb9484380a90127ef5a3e938a` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:31.472975 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e6d4db-09dd-4af6-bcc5-e391d84bc0f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ffffc6cb9484380a90127ef5a3e938a)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:31.475790 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e6d4db-09dd-4af6-bcc5-e391d84bc0f0 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:31.519483 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1086/2147] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:50:31 2022] POST /identity/v3/users => generated 284 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:50:31.523395 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0b06c9-1d1f-4c53-81f1-0f44daa98bdd admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:31.533953 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0b06c9-1d1f-4c53-81f1-0f44daa98bdd admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:31.539336 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52014853-6f2e-45ef-b488-80fb59eeb8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:31.539948 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52014853-6f2e-45ef-b488-80fb59eeb8a5 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:31.540233 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52014853-6f2e-45ef-b488-80fb59eeb8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:31.540567 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52014853-6f2e-45ef-b488-80fb59eeb8a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:31.541262 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52014853-6f2e-45ef-b488-80fb59eeb8a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:31.542859 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52014853-6f2e-45ef-b488-80fb59eeb8a5 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:31.554837 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1087/2148] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:50:31.559230 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52014853-6f2e-45ef-b488-80fb59eeb8a5 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:31.571371 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52014853-6f2e-45ef-b488-80fb59eeb8a5 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:31.576495 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc65ea0-017c-4ca8-8f81-1decc4b7b752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:31.577140 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc65ea0-017c-4ca8-8f81-1decc4b7b752 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:31.577420 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc65ea0-017c-4ca8-8f81-1decc4b7b752 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:31.577692 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc65ea0-017c-4ca8-8f81-1decc4b7b752 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:31.578331 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc65ea0-017c-4ca8-8f81-1decc4b7b752 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:31.579916 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc65ea0-017c-4ca8-8f81-1decc4b7b752 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:31.602634 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1088/2149] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:50:31.602634 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc65ea0-017c-4ca8-8f81-1decc4b7b752 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:31.606359 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc65ea0-017c-4ca8-8f81-1decc4b7b752 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:31.611031 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1715db5-f2e7-47f7-9be2-716792f6fe1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:31.611680 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1715db5-f2e7-47f7-9be2-716792f6fe1e admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:31.611957 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1715db5-f2e7-47f7-9be2-716792f6fe1e admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:31.612229 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1715db5-f2e7-47f7-9be2-716792f6fe1e admin admin] PATH_INFO: `/v3/projects/90e8bdd751634f0596fcd55fb1b4aa24/users/a45666139f9241898ec015bb83f362af/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:31.627987 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1715db5-f2e7-47f7-9be2-716792f6fe1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=90e8bdd751634f0596fcd55fb1b4aa24, user_id=a45666139f9241898ec015bb83f362af, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:31.631340 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1715db5-f2e7-47f7-9be2-716792f6fe1e admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:31.713561 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1089/2150] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:50:31 2022] PUT /identity/v3/projects/90e8bdd751634f0596fcd55fb1b4aa24/users/a45666139f9241898ec015bb83f362af/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:50:31.724452 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144fa51-309f-4154-ab5f-143b53b1c0eb None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:31.724742 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144fa51-309f-4154-ab5f-143b53b1c0eb None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:31.725045 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144fa51-309f-4154-ab5f-143b53b1c0eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:31.749776 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-78e6d4db-09dd-4af6-bcc5-e391d84bc0f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ffffc6cb9484380a90127ef5a3e938a'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:31.754453 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-f144fa51-309f-4154-ab5f-143b53b1c0eb None None] MFA Rules not processed for user `a45666139f9241898ec015bb83f362af`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:31.755506 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-f144fa51-309f-4154-ab5f-143b53b1c0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:31.762737 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-78e6d4db-09dd-4af6-bcc5-e391d84bc0f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ffffc6cb9484380a90127ef5a3e938a'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:31.772219 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-78e6d4db-09dd-4af6-bcc5-e391d84bc0f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ffffc6cb9484380a90127ef5a3e938a'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:31.835699 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1090/2151] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:50:31.849393 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144fa51-309f-4154-ab5f-143b53b1c0eb None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:31.850850 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-f144fa51-309f-4154-ab5f-143b53b1c0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:31.878612 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-78e6d4db-09dd-4af6-bcc5-e391d84bc0f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ffffc6cb9484380a90127ef5a3e938a'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:31.879279 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-78e6d4db-09dd-4af6-bcc5-e391d84bc0f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ffffc6cb9484380a90127ef5a3e938a'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:31.888657 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1062/2152] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:50:31 2022] DELETE /identity/v3/users/8ffffc6cb9484380a90127ef5a3e938a => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:50:31.929092 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144fa51-309f-4154-ab5f-143b53b1c0eb None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:31.969039 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6f185-16fe-4d82-b666-d82d808898ea None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:31.969039 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6f185-16fe-4d82-b666-d82d808898ea None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:31.969039 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6f185-16fe-4d82-b666-d82d808898ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:32.001227 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-67f6f185-16fe-4d82-b666-d82d808898ea None None] MFA Rules not processed for user `9c3c51d98e274150933a1306f7747f42`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:32.002671 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-67f6f185-16fe-4d82-b666-d82d808898ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:32.010528 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddbdda6-2a95-4e32-8804-2b964c2cdd2d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c2569e7cd774199aa5750fd6a850060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:32.015395 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddbdda6-2a95-4e32-8804-2b964c2cdd2d service glance] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:32.015395 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddbdda6-2a95-4e32-8804-2b964c2cdd2d service glance] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:32.015395 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddbdda6-2a95-4e32-8804-2b964c2cdd2d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:32.015395 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-6ddbdda6-2a95-4e32-8804-2b964c2cdd2d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:32.064480 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:32.085946 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1063/2153] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:50:32.090974 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddbdda6-2a95-4e32-8804-2b964c2cdd2d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:32.092423 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddbdda6-2a95-4e32-8804-2b964c2cdd2d service glance] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:32.097216 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f6f185-16fe-4d82-b666-d82d808898ea None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:32.099044 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-67f6f185-16fe-4d82-b666-d82d808898ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:32.103679 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1091/2154] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:50:31 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:50:32.169140 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f6f185-16fe-4d82-b666-d82d808898ea None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:32.177723 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32773fbe-9cab-4508-84c1-eb73fb2eb73f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:32.178340 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32773fbe-9cab-4508-84c1-eb73fb2eb73f service cinder] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:32.178618 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32773fbe-9cab-4508-84c1-eb73fb2eb73f service cinder] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:32.178894 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32773fbe-9cab-4508-84c1-eb73fb2eb73f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:32.188508 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32773fbe-9cab-4508-84c1-eb73fb2eb73f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:32.190195 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32773fbe-9cab-4508-84c1-eb73fb2eb73f service cinder] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:32.203684 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1064/2155] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:50:32 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:50:32.298232 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:32.353827 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddbdda6-2a95-4e32-8804-2b964c2cdd2d service glance] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:32.355233 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-6ddbdda6-2a95-4e32-8804-2b964c2cdd2d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:32.438149 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddbdda6-2a95-4e32-8804-2b964c2cdd2d service glance] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:32.443939 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c1828-bc09-4403-bd9c-d2826f708faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:32.444645 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c1828-bc09-4403-bd9c-d2826f708faf admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:32.444968 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c1828-bc09-4403-bd9c-d2826f708faf admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:32.451065 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c1828-bc09-4403-bd9c-d2826f708faf admin admin] PATH_INFO: `/v3/projects/5b7434a3f45f4bfea9d699069b12f988` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:32.451065 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088c1828-bc09-4403-bd9c-d2826f708faf admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b7434a3f45f4bfea9d699069b12f988)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:32.451065 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088c1828-bc09-4403-bd9c-d2826f708faf admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:32.643252 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-088c1828-bc09-4403-bd9c-d2826f708faf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b7434a3f45f4bfea9d699069b12f988', 'request_id': 'req-088c1828-bc09-4403-bd9c-d2826f708faf'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:32.650833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-088c1828-bc09-4403-bd9c-d2826f708faf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b7434a3f45f4bfea9d699069b12f988', 'request_id': 'req-088c1828-bc09-4403-bd9c-d2826f708faf'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:32.661060 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-088c1828-bc09-4403-bd9c-d2826f708faf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b7434a3f45f4bfea9d699069b12f988', 'request_id': 'req-088c1828-bc09-4403-bd9c-d2826f708faf'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:32.785489 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-088c1828-bc09-4403-bd9c-d2826f708faf admin admin] The token cache is being invalidate because project 5b7434a3f45f4bfea9d699069b12f988 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:50:32.785910 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-088c1828-bc09-4403-bd9c-d2826f708faf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:32.786660 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-088c1828-bc09-4403-bd9c-d2826f708faf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:32.788236 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1092/2156] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:50:32 2022] DELETE /identity/v3/projects/5b7434a3f45f4bfea9d699069b12f988 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:32.793119 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75ac5f-6990-4958-930e-4c82be1f1356 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:32.793596 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75ac5f-6990-4958-930e-4c82be1f1356 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:32.793691 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75ac5f-6990-4958-930e-4c82be1f1356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:32.824804 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-4a75ac5f-6990-4958-930e-4c82be1f1356 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:32.826045 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-4a75ac5f-6990-4958-930e-4c82be1f1356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:32.885304 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1065/2157] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:50:32.894692 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34efdae-8647-40f4-9d58-8043193e6a59 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:32.895039 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34efdae-8647-40f4-9d58-8043193e6a59 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:32.895318 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34efdae-8647-40f4-9d58-8043193e6a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:32.919449 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-e34efdae-8647-40f4-9d58-8043193e6a59 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:32.920665 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-e34efdae-8647-40f4-9d58-8043193e6a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:32.929944 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1093/2158] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:50:32.935399 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a75ac5f-6990-4958-930e-4c82be1f1356 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:32.948353 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a75ac5f-6990-4958-930e-4c82be1f1356 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:32.955075 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd4df9-ee6d-437b-94e1-e720ccaf2493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:32.955933 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd4df9-ee6d-437b-94e1-e720ccaf2493 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:32.956216 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd4df9-ee6d-437b-94e1-e720ccaf2493 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:32.956494 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd4df9-ee6d-437b-94e1-e720ccaf2493 admin admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:32.957908 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cd4df9-ee6d-437b-94e1-e720ccaf2493 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:50:32.958224 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cd4df9-ee6d-437b-94e1-e720ccaf2493 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:32.959830 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cd4df9-ee6d-437b-94e1-e720ccaf2493 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:32.972709 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1066/2159] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:50:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:50:32.979696 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34efdae-8647-40f4-9d58-8043193e6a59 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:32.992138 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34efdae-8647-40f4-9d58-8043193e6a59 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:32.997131 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a56fe3-3540-4a02-b5ed-c51bbc5eb5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:32.997855 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a56fe3-3540-4a02-b5ed-c51bbc5eb5d0 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:32.998140 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a56fe3-3540-4a02-b5ed-c51bbc5eb5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:32.998399 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a56fe3-3540-4a02-b5ed-c51bbc5eb5d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:32.999191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a56fe3-3540-4a02-b5ed-c51bbc5eb5d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:33.001401 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a56fe3-3540-4a02-b5ed-c51bbc5eb5d0 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:33.264391 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1094/2160] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:50:32 2022] POST /identity/v3/projects => generated 387 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:50:33.265549 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd4df9-ee6d-437b-94e1-e720ccaf2493 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:33.280987 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd4df9-ee6d-437b-94e1-e720ccaf2493 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:33.363478 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0ff81-c0de-4d7c-a988-bb6914bce042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:33.364289 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0ff81-c0de-4d7c-a988-bb6914bce042 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:33.364568 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0ff81-c0de-4d7c-a988-bb6914bce042 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:33.364871 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0ff81-c0de-4d7c-a988-bb6914bce042 admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:33.365500 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d0ff81-c0de-4d7c-a988-bb6914bce042 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:33.367489 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d0ff81-c0de-4d7c-a988-bb6914bce042 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:33.522317 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1067/2161] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:50:33 2022] POST /identity/v3/users => generated 294 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:50:33.525866 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a56fe3-3540-4a02-b5ed-c51bbc5eb5d0 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:33.547481 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a56fe3-3540-4a02-b5ed-c51bbc5eb5d0 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:33.555226 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095fe334-aba4-434b-b431-194ac8033654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:33.555992 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095fe334-aba4-434b-b431-194ac8033654 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:33.556330 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095fe334-aba4-434b-b431-194ac8033654 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:33.556663 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095fe334-aba4-434b-b431-194ac8033654 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:33.557476 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095fe334-aba4-434b-b431-194ac8033654 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:33.559195 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095fe334-aba4-434b-b431-194ac8033654 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:33.571101 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1095/2162] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:33 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:50:33.574903 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0ff81-c0de-4d7c-a988-bb6914bce042 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:33.585793 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0ff81-c0de-4d7c-a988-bb6914bce042 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:33.592528 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec75d8fd-93a5-40cf-9e87-38f34a74e647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:33.593717 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75d8fd-93a5-40cf-9e87-38f34a74e647 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:33.593717 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75d8fd-93a5-40cf-9e87-38f34a74e647 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:33.593717 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75d8fd-93a5-40cf-9e87-38f34a74e647 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:33.594334 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec75d8fd-93a5-40cf-9e87-38f34a74e647 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:33.597693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec75d8fd-93a5-40cf-9e87-38f34a74e647 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:33.616408 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1068/2163] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:33 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:50:33.620909 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095fe334-aba4-434b-b431-194ac8033654 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:33.633963 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095fe334-aba4-434b-b431-194ac8033654 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:33.642209 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1248f29-1575-423a-9dc5-49617f49f131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:33.643737 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1248f29-1575-423a-9dc5-49617f49f131 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:33.644024 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1248f29-1575-423a-9dc5-49617f49f131 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:33.644403 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1248f29-1575-423a-9dc5-49617f49f131 admin admin] PATH_INFO: `/v3/projects/b36b55388b384cc0885684a218675d9c/users/62bb7ccfcbf74e6fb382767070301f10/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:33.675137 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:33.698390 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1248f29-1575-423a-9dc5-49617f49f131 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b36b55388b384cc0885684a218675d9c, user_id=62bb7ccfcbf74e6fb382767070301f10, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:33.701466 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1248f29-1575-423a-9dc5-49617f49f131 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:33.739032 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:33.778399 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1096/2164] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:50:33 2022] PUT /identity/v3/projects/b36b55388b384cc0885684a218675d9c/users/62bb7ccfcbf74e6fb382767070301f10/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:33.790869 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c388c-6a2c-4d39-ba5f-5b27c4605e65 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:33.791403 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c388c-6a2c-4d39-ba5f-5b27c4605e65 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:33.791666 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c388c-6a2c-4d39-ba5f-5b27c4605e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:33.820134 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-283c388c-6a2c-4d39-ba5f-5b27c4605e65 None None] MFA Rules not processed for user `62bb7ccfcbf74e6fb382767070301f10`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:33.821380 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-283c388c-6a2c-4d39-ba5f-5b27c4605e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:33.889816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1069/2165] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:33 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:50:33.894708 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1248f29-1575-423a-9dc5-49617f49f131 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:33.963862 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1248f29-1575-423a-9dc5-49617f49f131 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:33.968854 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c86dd09-d7c6-448f-baec-b49e37bd62cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:33.969461 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86dd09-d7c6-448f-baec-b49e37bd62cd admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:33.969754 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86dd09-d7c6-448f-baec-b49e37bd62cd admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:33.970033 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86dd09-d7c6-448f-baec-b49e37bd62cd admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:33.970680 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c86dd09-d7c6-448f-baec-b49e37bd62cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:33.973173 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c86dd09-d7c6-448f-baec-b49e37bd62cd admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:34.066964 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1097/2166] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:50:33 2022] POST /identity/v3/projects => generated 387 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:50:34.071096 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c388c-6a2c-4d39-ba5f-5b27c4605e65 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:34.087024 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c388c-6a2c-4d39-ba5f-5b27c4605e65 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:34.154499 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ead62-5a41-4e7f-b512-58c2af48c335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:34.155105 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ead62-5a41-4e7f-b512-58c2af48c335 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:34.155381 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ead62-5a41-4e7f-b512-58c2af48c335 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:34.157872 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ead62-5a41-4e7f-b512-58c2af48c335 admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:34.157872 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12ead62-5a41-4e7f-b512-58c2af48c335 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:34.158405 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12ead62-5a41-4e7f-b512-58c2af48c335 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:34.252837 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1070/2167] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:50:34 2022] POST /identity/v3/users => generated 300 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:50:34.262380 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c86dd09-d7c6-448f-baec-b49e37bd62cd admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:34.279928 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c86dd09-d7c6-448f-baec-b49e37bd62cd admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:34.287033 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1890351a-041a-4734-b3e3-07e7973a017c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:34.287872 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1890351a-041a-4734-b3e3-07e7973a017c admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:34.288343 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1890351a-041a-4734-b3e3-07e7973a017c admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:34.288673 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1890351a-041a-4734-b3e3-07e7973a017c admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:34.289694 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1890351a-041a-4734-b3e3-07e7973a017c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:34.292460 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1890351a-041a-4734-b3e3-07e7973a017c admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:34.421309 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1098/2168] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:34 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:50:34.421820 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ead62-5a41-4e7f-b512-58c2af48c335 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:34.442949 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ead62-5a41-4e7f-b512-58c2af48c335 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:34.443450 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd12911-41dc-43c6-8e11-03c37bdfed0f None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:34.443738 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd12911-41dc-43c6-8e11-03c37bdfed0f None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:34.444020 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd12911-41dc-43c6-8e11-03c37bdfed0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:34.454945 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37bc8cc-ea7d-4fe2-a132-3768c97074cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:34.458024 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37bc8cc-ea7d-4fe2-a132-3768c97074cb admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:34.458317 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37bc8cc-ea7d-4fe2-a132-3768c97074cb admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:34.458600 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37bc8cc-ea7d-4fe2-a132-3768c97074cb admin admin] PATH_INFO: `/v3/projects/fa046193ae5342259c075f97135ed978/users/88fcb723e23a41658e8cd87f5b559967/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:34.481832 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-ecd12911-41dc-43c6-8e11-03c37bdfed0f None None] MFA Rules not processed for user `9c3c51d98e274150933a1306f7747f42`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:34.483083 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-ecd12911-41dc-43c6-8e11-03c37bdfed0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:34.484166 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37bc8cc-ea7d-4fe2-a132-3768c97074cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa046193ae5342259c075f97135ed978, user_id=88fcb723e23a41658e8cd87f5b559967, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:34.488417 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37bc8cc-ea7d-4fe2-a132-3768c97074cb admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:34.565315 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1099/2169] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:50:34.673408 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1071/2170] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:50:34 2022] PUT /identity/v3/projects/fa046193ae5342259c075f97135ed978/users/88fcb723e23a41658e8cd87f5b559967/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:34.677738 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd12911-41dc-43c6-8e11-03c37bdfed0f None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:34.691001 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd12911-41dc-43c6-8e11-03c37bdfed0f None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:34.786517 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf066fd2-341c-41c9-954f-42a7c72faa54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:34.787151 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf066fd2-341c-41c9-954f-42a7c72faa54 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:34.787429 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf066fd2-341c-41c9-954f-42a7c72faa54 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:34.787716 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf066fd2-341c-41c9-954f-42a7c72faa54 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:34.788356 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf066fd2-341c-41c9-954f-42a7c72faa54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:34.790016 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf066fd2-341c-41c9-954f-42a7c72faa54 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:34.803034 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1100/2171] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:34 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:50:34.807167 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37bc8cc-ea7d-4fe2-a132-3768c97074cb admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:34.823343 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37bc8cc-ea7d-4fe2-a132-3768c97074cb admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:34.828838 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e77ad-2f2f-4f87-badb-b2b4f8ffd4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:34.829504 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e77ad-2f2f-4f87-badb-b2b4f8ffd4f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:34.829851 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e77ad-2f2f-4f87-badb-b2b4f8ffd4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:34.830197 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e77ad-2f2f-4f87-badb-b2b4f8ffd4f4 admin admin] PATH_INFO: `/v3/projects/fa046193ae5342259c075f97135ed978/users/88fcb723e23a41658e8cd87f5b559967/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:34.834653 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60e77ad-2f2f-4f87-badb-b2b4f8ffd4f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa046193ae5342259c075f97135ed978, user_id=88fcb723e23a41658e8cd87f5b559967, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:34.837772 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60e77ad-2f2f-4f87-badb-b2b4f8ffd4f4 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:34.851802 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1072/2172] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:50:34 2022] PUT /identity/v3/projects/fa046193ae5342259c075f97135ed978/users/88fcb723e23a41658e8cd87f5b559967/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:34.863323 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec9628c-ab1a-49c9-b925-931e8d379812 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:34.864520 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec9628c-ab1a-49c9-b925-931e8d379812 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:34.864999 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec9628c-ab1a-49c9-b925-931e8d379812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:34.906867 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-aec9628c-ab1a-49c9-b925-931e8d379812 None None] MFA Rules not processed for user `88fcb723e23a41658e8cd87f5b559967`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:34.906867 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-aec9628c-ab1a-49c9-b925-931e8d379812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:35.056520 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1101/2173] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:34 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:50:35.083663 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e77ad-2f2f-4f87-badb-b2b4f8ffd4f4 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:35.086218 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-d60e77ad-2f2f-4f87-badb-b2b4f8ffd4f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:35.133621 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:35.248601 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e77ad-2f2f-4f87-badb-b2b4f8ffd4f4 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:35.261718 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af537437-d63f-433a-b7c9-26acd7aee752 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:35.262787 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af537437-d63f-433a-b7c9-26acd7aee752 service cinder] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:35.264294 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af537437-d63f-433a-b7c9-26acd7aee752 service cinder] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:35.268324 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af537437-d63f-433a-b7c9-26acd7aee752 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:35.268324 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:35.283771 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af537437-d63f-433a-b7c9-26acd7aee752 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:35.288492 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af537437-d63f-433a-b7c9-26acd7aee752 service cinder] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:35.326061 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1073/2174] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:50:35 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:50:35.456006 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec9628c-ab1a-49c9-b925-931e8d379812 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:35.568135 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec9628c-ab1a-49c9-b925-931e8d379812 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:35.575355 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc0711-0f8c-4a7b-907b-5ba8ec3cddbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:35.576423 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc0711-0f8c-4a7b-907b-5ba8ec3cddbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:35.576423 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc0711-0f8c-4a7b-907b-5ba8ec3cddbd admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:35.576787 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc0711-0f8c-4a7b-907b-5ba8ec3cddbd admin admin] PATH_INFO: `/v3/users/62bb7ccfcbf74e6fb382767070301f10` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:35.578192 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cc0711-0f8c-4a7b-907b-5ba8ec3cddbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=62bb7ccfcbf74e6fb382767070301f10)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:35.580190 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cc0711-0f8c-4a7b-907b-5ba8ec3cddbd admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:35.625071 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afefde19-ee51-4abc-985c-78d1e85a0c62 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:35.629885 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afefde19-ee51-4abc-985c-78d1e85a0c62 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:35.629885 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afefde19-ee51-4abc-985c-78d1e85a0c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:35.653814 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-afefde19-ee51-4abc-985c-78d1e85a0c62 None None] MFA Rules not processed for user `f77dfe78b1a8405084c06d9acee96511`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:35.655190 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-afefde19-ee51-4abc-985c-78d1e85a0c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:35.720494 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1074/2175] 10.222.0.52 () {58 vars in 1009 bytes} [Mon Aug 29 22:50:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:50:35.773912 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afefde19-ee51-4abc-985c-78d1e85a0c62 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:35.775339 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-afefde19-ee51-4abc-985c-78d1e85a0c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:35.836680 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-23cc0711-0f8c-4a7b-907b-5ba8ec3cddbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62bb7ccfcbf74e6fb382767070301f10'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:35.868537 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afefde19-ee51-4abc-985c-78d1e85a0c62 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:35.876399 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57872206-fbc0-4344-a77a-e5dff29c67d2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57c175ac90e545358746176972ff39a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:35.877058 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57872206-fbc0-4344-a77a-e5dff29c67d2 service swift] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:35.877374 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57872206-fbc0-4344-a77a-e5dff29c67d2 service swift] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:35.877651 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57872206-fbc0-4344-a77a-e5dff29c67d2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:35.887307 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57872206-fbc0-4344-a77a-e5dff29c67d2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:35.888939 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57872206-fbc0-4344-a77a-e5dff29c67d2 service swift] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:35.901233 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1075/2176] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Aug 29 22:50:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 29 22:50:35.927303 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-23cc0711-0f8c-4a7b-907b-5ba8ec3cddbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62bb7ccfcbf74e6fb382767070301f10'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:35.950736 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-23cc0711-0f8c-4a7b-907b-5ba8ec3cddbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62bb7ccfcbf74e6fb382767070301f10'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:35.982165 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-23cc0711-0f8c-4a7b-907b-5ba8ec3cddbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62bb7ccfcbf74e6fb382767070301f10'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:35.984424 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-23cc0711-0f8c-4a7b-907b-5ba8ec3cddbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62bb7ccfcbf74e6fb382767070301f10'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:36.004325 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1102/2177] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:50:35 2022] DELETE /identity/v3/users/62bb7ccfcbf74e6fb382767070301f10 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:36.008462 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd795b0c-3660-4ac2-8318-77f5c41c2b96 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:36.009002 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd795b0c-3660-4ac2-8318-77f5c41c2b96 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:36.009534 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd795b0c-3660-4ac2-8318-77f5c41c2b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:36.042035 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-dd795b0c-3660-4ac2-8318-77f5c41c2b96 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:36.043082 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-dd795b0c-3660-4ac2-8318-77f5c41c2b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:36.065642 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:36.161859 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1076/2178] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:50:36.169567 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:36.178058 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc0711-0f8c-4a7b-907b-5ba8ec3cddbd admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:36.180160 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-23cc0711-0f8c-4a7b-907b-5ba8ec3cddbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:36.264113 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc0711-0f8c-4a7b-907b-5ba8ec3cddbd admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:36.268812 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd1a7b1-2c52-4d25-9afc-ba4259b85a3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:36.272957 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1a7b1-2c52-4d25-9afc-ba4259b85a3c service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:36.273246 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1a7b1-2c52-4d25-9afc-ba4259b85a3c service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:36.273602 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1a7b1-2c52-4d25-9afc-ba4259b85a3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:36.288901 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd1a7b1-2c52-4d25-9afc-ba4259b85a3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:36.290464 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd1a7b1-2c52-4d25-9afc-ba4259b85a3c service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:36.308972 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1103/2179] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:50:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:50:36.568399 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd795b0c-3660-4ac2-8318-77f5c41c2b96 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:36.568399 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-dd795b0c-3660-4ac2-8318-77f5c41c2b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:36.593987 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd795b0c-3660-4ac2-8318-77f5c41c2b96 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:36.601775 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3560666-cc0a-4027-8812-a90a987fe5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:36.602468 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3560666-cc0a-4027-8812-a90a987fe5c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:36.602730 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3560666-cc0a-4027-8812-a90a987fe5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:36.603048 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3560666-cc0a-4027-8812-a90a987fe5c5 admin admin] PATH_INFO: `/v3/users/80449a09faeb424dad2e3470430584b7` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:36.604337 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3560666-cc0a-4027-8812-a90a987fe5c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80449a09faeb424dad2e3470430584b7)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:36.607263 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3560666-cc0a-4027-8812-a90a987fe5c5 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:36.900756 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-e3560666-cc0a-4027-8812-a90a987fe5c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80449a09faeb424dad2e3470430584b7'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:36.925776 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-e3560666-cc0a-4027-8812-a90a987fe5c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80449a09faeb424dad2e3470430584b7'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:36.952346 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-e3560666-cc0a-4027-8812-a90a987fe5c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80449a09faeb424dad2e3470430584b7'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:37.055493 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd1a7b1-2c52-4d25-9afc-ba4259b85a3c service neutron] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:37.056229 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-2bd1a7b1-2c52-4d25-9afc-ba4259b85a3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:37.077899 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-e3560666-cc0a-4027-8812-a90a987fe5c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80449a09faeb424dad2e3470430584b7'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:37.078987 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-e3560666-cc0a-4027-8812-a90a987fe5c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80449a09faeb424dad2e3470430584b7'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:37.089555 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd1a7b1-2c52-4d25-9afc-ba4259b85a3c service neutron] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:37.097448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1077/2180] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:50:36 2022] DELETE /identity/v3/users/80449a09faeb424dad2e3470430584b7 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:37.105121 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2299e1c-3a46-44b1-9add-3dbe2740bda8 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:37.105121 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2299e1c-3a46-44b1-9add-3dbe2740bda8 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:37.105121 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2299e1c-3a46-44b1-9add-3dbe2740bda8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:37.110759 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d1170d-0092-495e-8cd7-db9e8d7825b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:37.127123 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1170d-0092-495e-8cd7-db9e8d7825b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:37.127620 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1170d-0092-495e-8cd7-db9e8d7825b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:37.128065 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1170d-0092-495e-8cd7-db9e8d7825b7 admin admin] PATH_INFO: `/v3/projects/b36b55388b384cc0885684a218675d9c` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:37.132114 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d1170d-0092-495e-8cd7-db9e8d7825b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b36b55388b384cc0885684a218675d9c)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:37.134465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d1170d-0092-495e-8cd7-db9e8d7825b7 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:37.150857 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-e2299e1c-3a46-44b1-9add-3dbe2740bda8 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:37.152670 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-e2299e1c-3a46-44b1-9add-3dbe2740bda8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:37.277861 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1078/2181] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:50:37.327976 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2299e1c-3a46-44b1-9add-3dbe2740bda8 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:37.401698 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-e2299e1c-3a46-44b1-9add-3dbe2740bda8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:37.401698 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:37.402519 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-a6d1170d-0092-495e-8cd7-db9e8d7825b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b36b55388b384cc0885684a218675d9c', 'request_id': 'req-a6d1170d-0092-495e-8cd7-db9e8d7825b7'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:37.406536 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-a6d1170d-0092-495e-8cd7-db9e8d7825b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b36b55388b384cc0885684a218675d9c', 'request_id': 'req-a6d1170d-0092-495e-8cd7-db9e8d7825b7'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:37.439951 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-a6d1170d-0092-495e-8cd7-db9e8d7825b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b36b55388b384cc0885684a218675d9c', 'request_id': 'req-a6d1170d-0092-495e-8cd7-db9e8d7825b7'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:37.484949 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2299e1c-3a46-44b1-9add-3dbe2740bda8 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:37.574072 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa196bfc-b338-4be4-83a1-c04c622a554e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:37.574721 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa196bfc-b338-4be4-83a1-c04c622a554e service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:37.575004 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa196bfc-b338-4be4-83a1-c04c622a554e service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:37.575321 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa196bfc-b338-4be4-83a1-c04c622a554e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:37.682944 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa196bfc-b338-4be4-83a1-c04c622a554e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:37.684430 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa196bfc-b338-4be4-83a1-c04c622a554e service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:37.732821 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1079/2182] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:50:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:50:37.753661 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-a6d1170d-0092-495e-8cd7-db9e8d7825b7 admin admin] The token cache is being invalidate because project b36b55388b384cc0885684a218675d9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:50:37.754870 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-a6d1170d-0092-495e-8cd7-db9e8d7825b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:37.757092 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-a6d1170d-0092-495e-8cd7-db9e8d7825b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:37.760654 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1104/2183] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:50:37 2022] DELETE /identity/v3/projects/b36b55388b384cc0885684a218675d9c => generated 0 bytes in 707 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:37.767851 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa196bfc-b338-4be4-83a1-c04c622a554e service neutron] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:37.769568 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-aa196bfc-b338-4be4-83a1-c04c622a554e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:37.811460 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa196bfc-b338-4be4-83a1-c04c622a554e service neutron] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:37.822855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceda1391-9d2b-4c2e-a3ba-ac96f87cf535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:37.824364 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda1391-9d2b-4c2e-a3ba-ac96f87cf535 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:37.825476 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda1391-9d2b-4c2e-a3ba-ac96f87cf535 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:37.825934 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda1391-9d2b-4c2e-a3ba-ac96f87cf535 admin admin] PATH_INFO: `/v3/users/88fcb723e23a41658e8cd87f5b559967` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:37.831282 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceda1391-9d2b-4c2e-a3ba-ac96f87cf535 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88fcb723e23a41658e8cd87f5b559967)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:37.836159 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceda1391-9d2b-4c2e-a3ba-ac96f87cf535 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:38.475943 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-ceda1391-9d2b-4c2e-a3ba-ac96f87cf535 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88fcb723e23a41658e8cd87f5b559967'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:38.499493 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-ceda1391-9d2b-4c2e-a3ba-ac96f87cf535 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88fcb723e23a41658e8cd87f5b559967'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:38.509141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d1170d-0092-495e-8cd7-db9e8d7825b7 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:38.510739 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-a6d1170d-0092-495e-8cd7-db9e8d7825b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:38.518058 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-ceda1391-9d2b-4c2e-a3ba-ac96f87cf535 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88fcb723e23a41658e8cd87f5b559967'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:38.530781 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d1170d-0092-495e-8cd7-db9e8d7825b7 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:38.557499 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe104582-4894-4278-882a-2d8bf6b83d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:38.557499 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe104582-4894-4278-882a-2d8bf6b83d09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:38.557499 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe104582-4894-4278-882a-2d8bf6b83d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:38.557499 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe104582-4894-4278-882a-2d8bf6b83d09 admin admin] PATH_INFO: `/v3/projects/7e51e4fb4d424c9bb521b3f9e9e0b623` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:38.557911 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe104582-4894-4278-882a-2d8bf6b83d09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e51e4fb4d424c9bb521b3f9e9e0b623)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:38.562836 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe104582-4894-4278-882a-2d8bf6b83d09 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:38.610681 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-ceda1391-9d2b-4c2e-a3ba-ac96f87cf535 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88fcb723e23a41658e8cd87f5b559967'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:38.611537 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-ceda1391-9d2b-4c2e-a3ba-ac96f87cf535 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88fcb723e23a41658e8cd87f5b559967'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:38.619293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1080/2184] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:50:37 2022] DELETE /identity/v3/users/88fcb723e23a41658e8cd87f5b559967 => generated 0 bytes in 852 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:38.722762 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:38.745946 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-fe104582-4894-4278-882a-2d8bf6b83d09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e51e4fb4d424c9bb521b3f9e9e0b623', 'request_id': 'req-fe104582-4894-4278-882a-2d8bf6b83d09'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:38.758163 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-fe104582-4894-4278-882a-2d8bf6b83d09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e51e4fb4d424c9bb521b3f9e9e0b623', 'request_id': 'req-fe104582-4894-4278-882a-2d8bf6b83d09'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:38.766841 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-fe104582-4894-4278-882a-2d8bf6b83d09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e51e4fb4d424c9bb521b3f9e9e0b623', 'request_id': 'req-fe104582-4894-4278-882a-2d8bf6b83d09'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:38.920507 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-fe104582-4894-4278-882a-2d8bf6b83d09 admin admin] The token cache is being invalidate because project 7e51e4fb4d424c9bb521b3f9e9e0b623 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:50:38.920865 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-fe104582-4894-4278-882a-2d8bf6b83d09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:38.925641 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-fe104582-4894-4278-882a-2d8bf6b83d09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:38.929573 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1105/2185] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:50:38 2022] DELETE /identity/v3/projects/7e51e4fb4d424c9bb521b3f9e9e0b623 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:50:38.943632 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d5093-be34-4c74-a45a-8762c0a4b8aa None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:38.944952 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d5093-be34-4c74-a45a-8762c0a4b8aa None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:38.945652 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d5093-be34-4c74-a45a-8762c0a4b8aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:38.999200 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-ca5d5093-be34-4c74-a45a-8762c0a4b8aa None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:38.999200 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-ca5d5093-be34-4c74-a45a-8762c0a4b8aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:39.073417 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1081/2186] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:50:39.081538 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce1275-99ce-4777-9b7e-fe7c92b26206 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:39.087557 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce1275-99ce-4777-9b7e-fe7c92b26206 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:39.087557 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce1275-99ce-4777-9b7e-fe7c92b26206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:39.124537 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-7fce1275-99ce-4777-9b7e-fe7c92b26206 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:39.126070 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-7fce1275-99ce-4777-9b7e-fe7c92b26206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:39.137476 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1106/2187] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:50:39.142634 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d5093-be34-4c74-a45a-8762c0a4b8aa None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:39.171468 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d5093-be34-4c74-a45a-8762c0a4b8aa None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:39.181809 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda54679-79ca-4ca2-93c8-e6d71b5bd13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:39.183181 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda54679-79ca-4ca2-93c8-e6d71b5bd13b admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:39.183867 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda54679-79ca-4ca2-93c8-e6d71b5bd13b admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:39.184318 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda54679-79ca-4ca2-93c8-e6d71b5bd13b admin admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:39.185577 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda54679-79ca-4ca2-93c8-e6d71b5bd13b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:50:39.186081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda54679-79ca-4ca2-93c8-e6d71b5bd13b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:39.193371 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda54679-79ca-4ca2-93c8-e6d71b5bd13b admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:39.228931 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1082/2188] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:50:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:50:39.235077 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce1275-99ce-4777-9b7e-fe7c92b26206 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:39.275492 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce1275-99ce-4777-9b7e-fe7c92b26206 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:39.284736 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a4f72-11f4-499c-9aca-1f081b05dbd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:39.331636 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a4f72-11f4-499c-9aca-1f081b05dbd8 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:39.331636 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a4f72-11f4-499c-9aca-1f081b05dbd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:39.331636 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a4f72-11f4-499c-9aca-1f081b05dbd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:39.331636 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6a4f72-11f4-499c-9aca-1f081b05dbd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:39.331636 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6a4f72-11f4-499c-9aca-1f081b05dbd8 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:39.597542 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71729293-1b0f-4f3e-9809-851039ee4bca None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:39.598163 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71729293-1b0f-4f3e-9809-851039ee4bca None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:39.598163 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71729293-1b0f-4f3e-9809-851039ee4bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:39.623618 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1107/2189] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:50:39 2022] POST /identity/v3/projects => generated 383 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:50:39.628061 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a4f72-11f4-499c-9aca-1f081b05dbd8 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:39.632735 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-71729293-1b0f-4f3e-9809-851039ee4bca None None] MFA Rules not processed for user `53f9f8eb7a20431ca0d679af45946bce`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:39.634260 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-71729293-1b0f-4f3e-9809-851039ee4bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:39.736391 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1083/2190] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:39 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:50:39.736391 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71729293-1b0f-4f3e-9809-851039ee4bca None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:39.738632 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-71729293-1b0f-4f3e-9809-851039ee4bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:39.812113 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a4f72-11f4-499c-9aca-1f081b05dbd8 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:39.819638 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62813d6-6767-41a3-b86a-51a68b1a056b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:39.820543 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62813d6-6767-41a3-b86a-51a68b1a056b admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:39.821216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62813d6-6767-41a3-b86a-51a68b1a056b admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:39.821923 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62813d6-6767-41a3-b86a-51a68b1a056b admin admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:39.824191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62813d6-6767-41a3-b86a-51a68b1a056b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:39.828987 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62813d6-6767-41a3-b86a-51a68b1a056b admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:39.885368 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71729293-1b0f-4f3e-9809-851039ee4bca None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:39.885368 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0af7fc8-ab91-4fdf-81d3-fe84295cf35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:39.885368 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af7fc8-ab91-4fdf-81d3-fe84295cf35c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:39.885368 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af7fc8-ab91-4fdf-81d3-fe84295cf35c admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:39.885927 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af7fc8-ab91-4fdf-81d3-fe84295cf35c admin admin] PATH_INFO: `/v3/projects/fa046193ae5342259c075f97135ed978` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:39.885927 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0af7fc8-ab91-4fdf-81d3-fe84295cf35c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa046193ae5342259c075f97135ed978)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:39.885927 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0af7fc8-ab91-4fdf-81d3-fe84295cf35c admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:40.195984 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1108/2191] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:50:39 2022] POST /identity/v3/users => generated 292 bytes in 568 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:50:40.199598 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62813d6-6767-41a3-b86a-51a68b1a056b admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:40.210858 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62813d6-6767-41a3-b86a-51a68b1a056b admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:40.215513 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad7ef23-6bba-4f6f-ac81-8e0ea70a7cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:40.216133 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7ef23-6bba-4f6f-ac81-8e0ea70a7cde admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:40.216471 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7ef23-6bba-4f6f-ac81-8e0ea70a7cde admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:40.216778 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7ef23-6bba-4f6f-ac81-8e0ea70a7cde admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:40.217432 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad7ef23-6bba-4f6f-ac81-8e0ea70a7cde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:40.219037 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad7ef23-6bba-4f6f-ac81-8e0ea70a7cde admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:40.248139 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1109/2192] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:40 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:50:40.250779 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-c0af7fc8-ab91-4fdf-81d3-fe84295cf35c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa046193ae5342259c075f97135ed978', 'request_id': 'req-c0af7fc8-ab91-4fdf-81d3-fe84295cf35c'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:40.252765 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad7ef23-6bba-4f6f-ac81-8e0ea70a7cde admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:40.264674 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad7ef23-6bba-4f6f-ac81-8e0ea70a7cde admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:40.343473 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80c53cc-54c9-4c87-b137-ff64b9402081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:40.344586 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c53cc-54c9-4c87-b137-ff64b9402081 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:40.345105 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c53cc-54c9-4c87-b137-ff64b9402081 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:40.345615 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c53cc-54c9-4c87-b137-ff64b9402081 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:40.346283 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80c53cc-54c9-4c87-b137-ff64b9402081 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:40.348242 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80c53cc-54c9-4c87-b137-ff64b9402081 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:40.364012 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1110/2193] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:40 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:50:40.368852 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80c53cc-54c9-4c87-b137-ff64b9402081 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:40.391841 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-c0af7fc8-ab91-4fdf-81d3-fe84295cf35c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa046193ae5342259c075f97135ed978', 'request_id': 'req-c0af7fc8-ab91-4fdf-81d3-fe84295cf35c'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:40.402480 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:40.409534 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-c0af7fc8-ab91-4fdf-81d3-fe84295cf35c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa046193ae5342259c075f97135ed978', 'request_id': 'req-c0af7fc8-ab91-4fdf-81d3-fe84295cf35c'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:40.409697 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80c53cc-54c9-4c87-b137-ff64b9402081 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:40.424776 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3f6c1-47d6-4957-8f77-517948ff5f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:40.426959 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3f6c1-47d6-4957-8f77-517948ff5f52 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:40.428794 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3f6c1-47d6-4957-8f77-517948ff5f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:40.429189 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3f6c1-47d6-4957-8f77-517948ff5f52 admin admin] PATH_INFO: `/v3/projects/f3de8648480447988acfa6f0652ff86d/users/0518929bc0024ac58ecd2a24a81769e1/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:40.432306 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-c0af7fc8-ab91-4fdf-81d3-fe84295cf35c admin admin] The token cache is being invalidate because project fa046193ae5342259c075f97135ed978 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:50:40.432921 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-c0af7fc8-ab91-4fdf-81d3-fe84295cf35c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:40.442128 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-c0af7fc8-ab91-4fdf-81d3-fe84295cf35c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:40.455133 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1084/2194] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:50:39 2022] DELETE /identity/v3/projects/fa046193ae5342259c075f97135ed978 => generated 0 bytes in 723 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:40.467421 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e812d9-b6de-4b9a-b6d1-f9e5be3b7a65 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:40.467949 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e812d9-b6de-4b9a-b6d1-f9e5be3b7a65 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:40.469818 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e812d9-b6de-4b9a-b6d1-f9e5be3b7a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:40.537424 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a3f6c1-47d6-4957-8f77-517948ff5f52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3de8648480447988acfa6f0652ff86d, user_id=0518929bc0024ac58ecd2a24a81769e1, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:40.540355 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a3f6c1-47d6-4957-8f77-517948ff5f52 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:40.563187 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-14e812d9-b6de-4b9a-b6d1-f9e5be3b7a65 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:40.565896 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-14e812d9-b6de-4b9a-b6d1-f9e5be3b7a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:40.593081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1085/2195] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:50:40.595963 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8947b866-3428-42bc-ba4a-e16029b51492 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:40.595963 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8947b866-3428-42bc-ba4a-e16029b51492 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:40.596294 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8947b866-3428-42bc-ba4a-e16029b51492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:40.630846 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-8947b866-3428-42bc-ba4a-e16029b51492 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:40.634868 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-8947b866-3428-42bc-ba4a-e16029b51492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:40.681924 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1086/2196] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:50:40.685032 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8947b866-3428-42bc-ba4a-e16029b51492 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:40.703510 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1111/2197] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:50:40 2022] PUT /identity/v3/projects/f3de8648480447988acfa6f0652ff86d/users/0518929bc0024ac58ecd2a24a81769e1/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:40.722230 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51fea81-f20b-4de8-8547-df400d52c1e4 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:40.722230 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51fea81-f20b-4de8-8547-df400d52c1e4 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:40.722484 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8947b866-3428-42bc-ba4a-e16029b51492 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:40.722618 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51fea81-f20b-4de8-8547-df400d52c1e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:40.814269 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-f51fea81-f20b-4de8-8547-df400d52c1e4 None None] MFA Rules not processed for user `0518929bc0024ac58ecd2a24a81769e1`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:40.817513 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-f51fea81-f20b-4de8-8547-df400d52c1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:40.898497 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3f7f22-81f4-4e9e-871d-4165ccf632b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:40.900930 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3f7f22-81f4-4e9e-871d-4165ccf632b4 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:40.904285 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3f7f22-81f4-4e9e-871d-4165ccf632b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:40.904597 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3f7f22-81f4-4e9e-871d-4165ccf632b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:40.905336 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3f7f22-81f4-4e9e-871d-4165ccf632b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:50:40.911646 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3f7f22-81f4-4e9e-871d-4165ccf632b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:40.911646 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3f7f22-81f4-4e9e-871d-4165ccf632b4 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:40.956278 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1087/2198] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:50:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:50:40.965453 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3f7f22-81f4-4e9e-871d-4165ccf632b4 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:40.981466 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3f7f22-81f4-4e9e-871d-4165ccf632b4 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:40.999045 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d9e528-e986-46b3-92f9-19336c6a4fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:40.999777 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d9e528-e986-46b3-92f9-19336c6a4fb9 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:41.000077 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d9e528-e986-46b3-92f9-19336c6a4fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:41.000638 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d9e528-e986-46b3-92f9-19336c6a4fb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:41.001924 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d9e528-e986-46b3-92f9-19336c6a4fb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:41.005248 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d9e528-e986-46b3-92f9-19336c6a4fb9 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:41.011976 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1112/2199] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:50:41.023749 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51fea81-f20b-4de8-8547-df400d52c1e4 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:41.031952 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-f51fea81-f20b-4de8-8547-df400d52c1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:41.106850 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:41.149242 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1088/2200] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:50:40 2022] POST /identity/v3/projects => generated 361 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:50:41.150470 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d9e528-e986-46b3-92f9-19336c6a4fb9 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:41.150873 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51fea81-f20b-4de8-8547-df400d52c1e4 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:41.269110 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d9e528-e986-46b3-92f9-19336c6a4fb9 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:41.276533 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ac7cf-a789-4141-bfdb-1ce076f07d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:41.277450 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ac7cf-a789-4141-bfdb-1ce076f07d7a admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:41.283720 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ac7cf-a789-4141-bfdb-1ce076f07d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:41.283720 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ac7cf-a789-4141-bfdb-1ce076f07d7a admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:41.283720 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8ac7cf-a789-4141-bfdb-1ce076f07d7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:41.283720 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8ac7cf-a789-4141-bfdb-1ce076f07d7a admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:41.288677 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04efe133-e907-41d0-8356-27343073230f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:41.289427 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04efe133-e907-41d0-8356-27343073230f service cinder] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:41.289890 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04efe133-e907-41d0-8356-27343073230f service cinder] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:41.290343 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04efe133-e907-41d0-8356-27343073230f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:41.415378 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04efe133-e907-41d0-8356-27343073230f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:41.417550 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04efe133-e907-41d0-8356-27343073230f service cinder] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:41.423193 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1089/2201] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:50:41 2022] POST /identity/v3/users => generated 281 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:50:41.429496 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ac7cf-a789-4141-bfdb-1ce076f07d7a admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:41.437460 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1113/2202] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:50:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:50:41.456687 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ac7cf-a789-4141-bfdb-1ce076f07d7a admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:41.472365 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:INSERT=1,keystone:DELETE=2,keystone:SELECT=97 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:41.483009 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be9636-a56a-4847-8762-0d887dd25718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:41.483543 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be9636-a56a-4847-8762-0d887dd25718 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:41.483824 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be9636-a56a-4847-8762-0d887dd25718 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:41.485839 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be9636-a56a-4847-8762-0d887dd25718 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:41.486790 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0be9636-a56a-4847-8762-0d887dd25718 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:41.502163 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0be9636-a56a-4847-8762-0d887dd25718 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:41.508913 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1090/2203] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:41 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:50:41.513332 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04efe133-e907-41d0-8356-27343073230f service cinder] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:41.552743 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04efe133-e907-41d0-8356-27343073230f service cinder] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:41.563107 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd76d4a9-867c-4cb8-9925-0008033f973f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:41.563748 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76d4a9-867c-4cb8-9925-0008033f973f admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:41.564030 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76d4a9-867c-4cb8-9925-0008033f973f admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:41.564301 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76d4a9-867c-4cb8-9925-0008033f973f admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:41.565067 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd76d4a9-867c-4cb8-9925-0008033f973f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:41.566687 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd76d4a9-867c-4cb8-9925-0008033f973f admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:41.605053 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1114/2204] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:41 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:50:41.615535 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be9636-a56a-4847-8762-0d887dd25718 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:41.634089 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be9636-a56a-4847-8762-0d887dd25718 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:41.647622 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14920ecf-6328-4dc6-a265-8a1b6a21e4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:41.649862 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14920ecf-6328-4dc6-a265-8a1b6a21e4cc admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:41.650312 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14920ecf-6328-4dc6-a265-8a1b6a21e4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:41.650812 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14920ecf-6328-4dc6-a265-8a1b6a21e4cc admin admin] PATH_INFO: `/v3/projects/59d9161e0aa448c6b737de1a10c41e4e/users/cbc237ecb45f4d9baafe2e53b445cf59/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:41.678980 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14920ecf-6328-4dc6-a265-8a1b6a21e4cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=59d9161e0aa448c6b737de1a10c41e4e, user_id=cbc237ecb45f4d9baafe2e53b445cf59, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:41.682029 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14920ecf-6328-4dc6-a265-8a1b6a21e4cc admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:41.791203 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1091/2205] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:50:41 2022] PUT /identity/v3/projects/59d9161e0aa448c6b737de1a10c41e4e/users/cbc237ecb45f4d9baafe2e53b445cf59/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:41.801797 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64868914-ac0c-4558-94ae-e767d9689d38 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:41.802531 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64868914-ac0c-4558-94ae-e767d9689d38 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:41.802531 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64868914-ac0c-4558-94ae-e767d9689d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:41.863538 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-64868914-ac0c-4558-94ae-e767d9689d38 None None] MFA Rules not processed for user `cbc237ecb45f4d9baafe2e53b445cf59`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:41.866300 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-64868914-ac0c-4558-94ae-e767d9689d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:41.985585 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1115/2206] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:50:41.986868 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14920ecf-6328-4dc6-a265-8a1b6a21e4cc admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:42.142883 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14920ecf-6328-4dc6-a265-8a1b6a21e4cc admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:42.162849 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3fcb03-af5f-46e3-994f-699835cd040c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:42.164247 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3fcb03-af5f-46e3-994f-699835cd040c admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:42.164685 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3fcb03-af5f-46e3-994f-699835cd040c admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:42.165218 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3fcb03-af5f-46e3-994f-699835cd040c admin admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:42.167729 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3fcb03-af5f-46e3-994f-699835cd040c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:42.170948 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3fcb03-af5f-46e3-994f-699835cd040c admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:42.275442 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1092/2207] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:50:41 2022] POST /identity/v3/projects => generated 367 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:50:42.276970 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64868914-ac0c-4558-94ae-e767d9689d38 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:42.301661 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64868914-ac0c-4558-94ae-e767d9689d38 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:42.492521 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8a6c1c-6617-4959-be90-a326cf713aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:42.493225 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a6c1c-6617-4959-be90-a326cf713aa3 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:42.493463 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a6c1c-6617-4959-be90-a326cf713aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:42.493741 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a6c1c-6617-4959-be90-a326cf713aa3 admin admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:42.494387 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8a6c1c-6617-4959-be90-a326cf713aa3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:42.496925 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8a6c1c-6617-4959-be90-a326cf713aa3 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:42.606400 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1116/2208] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:50:42 2022] POST /identity/v3/users => generated 290 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:50:42.610601 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3fcb03-af5f-46e3-994f-699835cd040c admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:42.627491 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3fcb03-af5f-46e3-994f-699835cd040c admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:42.648168 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:42.655417 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdfb3a1-166f-4316-b2de-4ed85251541e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:42.656673 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdfb3a1-166f-4316-b2de-4ed85251541e admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:42.660506 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdfb3a1-166f-4316-b2de-4ed85251541e admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:42.661015 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdfb3a1-166f-4316-b2de-4ed85251541e admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:42.662268 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdfb3a1-166f-4316-b2de-4ed85251541e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:42.666039 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdfb3a1-166f-4316-b2de-4ed85251541e admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:42.697158 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1093/2209] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:42 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:50:42.697297 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8a6c1c-6617-4959-be90-a326cf713aa3 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:42.709320 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8a6c1c-6617-4959-be90-a326cf713aa3 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:42.715452 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2fbabc-262f-4d0e-b4d5-e15ae39d03c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:42.716292 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2fbabc-262f-4d0e-b4d5-e15ae39d03c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:42.716672 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2fbabc-262f-4d0e-b4d5-e15ae39d03c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:42.717072 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2fbabc-262f-4d0e-b4d5-e15ae39d03c5 admin admin] PATH_INFO: `/v3/projects/54c6cd633f104c75b4db739ef5b710e5/users/f8f0020ced3b4b71b88458f7c883cbf5/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:42.742514 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2fbabc-262f-4d0e-b4d5-e15ae39d03c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54c6cd633f104c75b4db739ef5b710e5, user_id=f8f0020ced3b4b71b88458f7c883cbf5, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:42.745966 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2fbabc-262f-4d0e-b4d5-e15ae39d03c5 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:42.831747 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1117/2210] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:50:42 2022] PUT /identity/v3/projects/54c6cd633f104c75b4db739ef5b710e5/users/f8f0020ced3b4b71b88458f7c883cbf5/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:50:42.836047 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdfb3a1-166f-4316-b2de-4ed85251541e admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:42.849436 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdfb3a1-166f-4316-b2de-4ed85251541e admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:42.956903 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:42.959909 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d58f3-f482-47c4-9239-41292ed46edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:42.961309 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d58f3-f482-47c4-9239-41292ed46edf admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:42.961545 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d58f3-f482-47c4-9239-41292ed46edf admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:42.962162 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d58f3-f482-47c4-9239-41292ed46edf admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:42.962823 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731d58f3-f482-47c4-9239-41292ed46edf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:42.965090 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731d58f3-f482-47c4-9239-41292ed46edf admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:42.982827 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1094/2211] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:42 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:50:42.987168 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2fbabc-262f-4d0e-b4d5-e15ae39d03c5 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:43.020381 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2fbabc-262f-4d0e-b4d5-e15ae39d03c5 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:43.026500 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447a96c6-d22e-4d69-9f97-c1ec96645882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:43.027187 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a96c6-d22e-4d69-9f97-c1ec96645882 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:43.027500 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a96c6-d22e-4d69-9f97-c1ec96645882 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:43.027775 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a96c6-d22e-4d69-9f97-c1ec96645882 admin admin] PATH_INFO: `/v3/projects/54c6cd633f104c75b4db739ef5b710e5/users/f8f0020ced3b4b71b88458f7c883cbf5/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:43.029776 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447a96c6-d22e-4d69-9f97-c1ec96645882 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54c6cd633f104c75b4db739ef5b710e5, user_id=f8f0020ced3b4b71b88458f7c883cbf5, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:43.033260 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447a96c6-d22e-4d69-9f97-c1ec96645882 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:43.069366 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1118/2212] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:50:42 2022] PUT /identity/v3/projects/54c6cd633f104c75b4db739ef5b710e5/users/f8f0020ced3b4b71b88458f7c883cbf5/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:43.085675 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5153617-af77-42a5-9079-3714aa96677d None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:43.085675 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5153617-af77-42a5-9079-3714aa96677d None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:43.085675 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5153617-af77-42a5-9079-3714aa96677d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:43.117375 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-e5153617-af77-42a5-9079-3714aa96677d None None] MFA Rules not processed for user `f8f0020ced3b4b71b88458f7c883cbf5`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:43.119327 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-e5153617-af77-42a5-9079-3714aa96677d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:43.231193 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1095/2213] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:43 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:50:43.259306 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447a96c6-d22e-4d69-9f97-c1ec96645882 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:43.501771 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447a96c6-d22e-4d69-9f97-c1ec96645882 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:43.510986 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f456df-e7ba-4f96-811e-156717a9aedc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:43.511677 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f456df-e7ba-4f96-811e-156717a9aedc service cinder] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:43.511965 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f456df-e7ba-4f96-811e-156717a9aedc service cinder] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:43.512242 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f456df-e7ba-4f96-811e-156717a9aedc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:43.541055 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f456df-e7ba-4f96-811e-156717a9aedc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:43.542070 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f456df-e7ba-4f96-811e-156717a9aedc service cinder] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:43.594137 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1119/2214] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:50:43 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:50:43.618068 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:43.934487 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5153617-af77-42a5-9079-3714aa96677d None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:43.949638 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5153617-af77-42a5-9079-3714aa96677d None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:43.962141 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff85bbf0-d1bf-412a-b790-4da8b3e5246d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:43.962949 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85bbf0-d1bf-412a-b790-4da8b3e5246d service cinder] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:43.963744 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85bbf0-d1bf-412a-b790-4da8b3e5246d service cinder] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:43.964113 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85bbf0-d1bf-412a-b790-4da8b3e5246d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:44.064183 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff85bbf0-d1bf-412a-b790-4da8b3e5246d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:44.066094 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff85bbf0-d1bf-412a-b790-4da8b3e5246d service cinder] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:44.079881 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1096/2215] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:50:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:50:44.289885 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:53.773962 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f456df-e7ba-4f96-811e-156717a9aedc service cinder] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:53.778229 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-60f456df-e7ba-4f96-811e-156717a9aedc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:53.853844 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f456df-e7ba-4f96-811e-156717a9aedc service cinder] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:53.859684 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce627488-8c90-4d61-8e17-355e5952f25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:53.860397 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce627488-8c90-4d61-8e17-355e5952f25a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:53.860678 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce627488-8c90-4d61-8e17-355e5952f25a admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:53.861103 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce627488-8c90-4d61-8e17-355e5952f25a admin admin] PATH_INFO: `/v3/users/0518929bc0024ac58ecd2a24a81769e1` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:53.862535 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce627488-8c90-4d61-8e17-355e5952f25a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0518929bc0024ac58ecd2a24a81769e1)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:53.876756 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce627488-8c90-4d61-8e17-355e5952f25a admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:53.963483 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff85bbf0-d1bf-412a-b790-4da8b3e5246d service cinder] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:53.975396 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff85bbf0-d1bf-412a-b790-4da8b3e5246d service cinder] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:53.980279 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fb2140-fff1-48d5-a0f8-6df52944e70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:53.980982 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fb2140-fff1-48d5-a0f8-6df52944e70f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:53.981270 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fb2140-fff1-48d5-a0f8-6df52944e70f admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:53.981549 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fb2140-fff1-48d5-a0f8-6df52944e70f admin admin] PATH_INFO: `/v3/users/cbc237ecb45f4d9baafe2e53b445cf59` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:53.982726 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fb2140-fff1-48d5-a0f8-6df52944e70f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbc237ecb45f4d9baafe2e53b445cf59)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:53.984636 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fb2140-fff1-48d5-a0f8-6df52944e70f admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:54.097347 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ce627488-8c90-4d61-8e17-355e5952f25a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0518929bc0024ac58ecd2a24a81769e1'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:54.210942 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ce627488-8c90-4d61-8e17-355e5952f25a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0518929bc0024ac58ecd2a24a81769e1'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:54.221189 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-c3fb2140-fff1-48d5-a0f8-6df52944e70f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbc237ecb45f4d9baafe2e53b445cf59'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:54.231139 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ce627488-8c90-4d61-8e17-355e5952f25a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0518929bc0024ac58ecd2a24a81769e1'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:54.238433 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ce627488-8c90-4d61-8e17-355e5952f25a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0518929bc0024ac58ecd2a24a81769e1'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:54.239779 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ce627488-8c90-4d61-8e17-355e5952f25a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0518929bc0024ac58ecd2a24a81769e1'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:54.246047 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1120/2216] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:50:53 2022] DELETE /identity/v3/users/0518929bc0024ac58ecd2a24a81769e1 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:54.249527 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-c3fb2140-fff1-48d5-a0f8-6df52944e70f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbc237ecb45f4d9baafe2e53b445cf59'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:54.250380 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c166b4-400f-456e-98d6-11d440831361 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:54.250863 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c166b4-400f-456e-98d6-11d440831361 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:54.251204 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c166b4-400f-456e-98d6-11d440831361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:54.257861 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-c3fb2140-fff1-48d5-a0f8-6df52944e70f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbc237ecb45f4d9baafe2e53b445cf59'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:54.271987 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-67c166b4-400f-456e-98d6-11d440831361 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:54.273316 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-67c166b4-400f-456e-98d6-11d440831361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:54.311587 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:54.332973 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1121/2217] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:50:54.334954 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-c3fb2140-fff1-48d5-a0f8-6df52944e70f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbc237ecb45f4d9baafe2e53b445cf59'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:54.335667 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-c3fb2140-fff1-48d5-a0f8-6df52944e70f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbc237ecb45f4d9baafe2e53b445cf59'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:54.342821 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1097/2218] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:50:53 2022] DELETE /identity/v3/users/cbc237ecb45f4d9baafe2e53b445cf59 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:50:54.343121 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c166b4-400f-456e-98d6-11d440831361 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:54.344286 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-67c166b4-400f-456e-98d6-11d440831361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:54.347136 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1001a860-649f-44c1-908e-bf415acda290 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:54.347724 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1001a860-649f-44c1-908e-bf415acda290 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:54.348160 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1001a860-649f-44c1-908e-bf415acda290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:54.368777 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-1001a860-649f-44c1-908e-bf415acda290 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:54.369985 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-1001a860-649f-44c1-908e-bf415acda290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:54.402605 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:54.405601 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c166b4-400f-456e-98d6-11d440831361 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:54.434810 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1098/2219] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:50:54.444669 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1001a860-649f-44c1-908e-bf415acda290 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:54.446164 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-1001a860-649f-44c1-908e-bf415acda290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:54.461151 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33861d8e-3511-4a86-9d8e-eeeda69da86d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:54.461786 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33861d8e-3511-4a86-9d8e-eeeda69da86d service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:54.462088 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33861d8e-3511-4a86-9d8e-eeeda69da86d service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:54.462346 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33861d8e-3511-4a86-9d8e-eeeda69da86d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:54.464341 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-33861d8e-3511-4a86-9d8e-eeeda69da86d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:54.477511 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33861d8e-3511-4a86-9d8e-eeeda69da86d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:54.478934 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33861d8e-3511-4a86-9d8e-eeeda69da86d service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:54.490662 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1122/2220] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:50:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:50:54.505051 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1001a860-649f-44c1-908e-bf415acda290 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:54.509016 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be8d72-86f9-4acb-ab9b-54a8cd706afd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:54.509775 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be8d72-86f9-4acb-ab9b-54a8cd706afd service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:54.510468 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be8d72-86f9-4acb-ab9b-54a8cd706afd service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:54.510865 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be8d72-86f9-4acb-ab9b-54a8cd706afd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:54.520406 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74be8d72-86f9-4acb-ab9b-54a8cd706afd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:54.521886 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74be8d72-86f9-4acb-ab9b-54a8cd706afd service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:54.532645 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1099/2221] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:50:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:50:55.465874 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33861d8e-3511-4a86-9d8e-eeeda69da86d service neutron] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:55.467467 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-33861d8e-3511-4a86-9d8e-eeeda69da86d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:55.470375 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be8d72-86f9-4acb-ab9b-54a8cd706afd service neutron] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:55.472631 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-74be8d72-86f9-4acb-ab9b-54a8cd706afd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:55.488384 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be8d72-86f9-4acb-ab9b-54a8cd706afd service neutron] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:55.491420 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33861d8e-3511-4a86-9d8e-eeeda69da86d service neutron] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:55.494482 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab476b3-aa37-4c4c-870d-4a0c2a276de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:55.495501 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab476b3-aa37-4c4c-870d-4a0c2a276de4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:55.496144 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab476b3-aa37-4c4c-870d-4a0c2a276de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:55.496631 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab476b3-aa37-4c4c-870d-4a0c2a276de4 admin admin] PATH_INFO: `/v3/projects/f3de8648480447988acfa6f0652ff86d` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:55.498119 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab476b3-aa37-4c4c-870d-4a0c2a276de4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3de8648480447988acfa6f0652ff86d)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:55.500239 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab476b3-aa37-4c4c-870d-4a0c2a276de4 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:55.513079 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c1479-a7c9-42d4-989f-fd25dac1d540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:55.513626 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c1479-a7c9-42d4-989f-fd25dac1d540 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:55.513911 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c1479-a7c9-42d4-989f-fd25dac1d540 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:55.514180 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c1479-a7c9-42d4-989f-fd25dac1d540 admin admin] PATH_INFO: `/v3/projects/59d9161e0aa448c6b737de1a10c41e4e` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:55.515749 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63c1479-a7c9-42d4-989f-fd25dac1d540 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59d9161e0aa448c6b737de1a10c41e4e)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:55.517907 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63c1479-a7c9-42d4-989f-fd25dac1d540 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:55.636044 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-aab476b3-aa37-4c4c-870d-4a0c2a276de4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3de8648480447988acfa6f0652ff86d', 'request_id': 'req-aab476b3-aa37-4c4c-870d-4a0c2a276de4'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:55.663849 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-b63c1479-a7c9-42d4-989f-fd25dac1d540 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59d9161e0aa448c6b737de1a10c41e4e', 'request_id': 'req-b63c1479-a7c9-42d4-989f-fd25dac1d540'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:55.669702 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-b63c1479-a7c9-42d4-989f-fd25dac1d540 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59d9161e0aa448c6b737de1a10c41e4e', 'request_id': 'req-b63c1479-a7c9-42d4-989f-fd25dac1d540'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:55.676290 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-b63c1479-a7c9-42d4-989f-fd25dac1d540 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59d9161e0aa448c6b737de1a10c41e4e', 'request_id': 'req-b63c1479-a7c9-42d4-989f-fd25dac1d540'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:55.726430 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-aab476b3-aa37-4c4c-870d-4a0c2a276de4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3de8648480447988acfa6f0652ff86d', 'request_id': 'req-aab476b3-aa37-4c4c-870d-4a0c2a276de4'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:55.732972 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-aab476b3-aa37-4c4c-870d-4a0c2a276de4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3de8648480447988acfa6f0652ff86d', 'request_id': 'req-aab476b3-aa37-4c4c-870d-4a0c2a276de4'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:55.748445 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-aab476b3-aa37-4c4c-870d-4a0c2a276de4 admin admin] The token cache is being invalidate because project f3de8648480447988acfa6f0652ff86d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:50:55.749003 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-aab476b3-aa37-4c4c-870d-4a0c2a276de4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:55.750142 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-aab476b3-aa37-4c4c-870d-4a0c2a276de4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:55.752082 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1100/2222] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:50:55 2022] DELETE /identity/v3/projects/f3de8648480447988acfa6f0652ff86d => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:50:55.758014 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66be3a6-449e-4b54-91bf-fa20b583c990 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:55.758364 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66be3a6-449e-4b54-91bf-fa20b583c990 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:55.758681 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66be3a6-449e-4b54-91bf-fa20b583c990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:55.778678 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-d66be3a6-449e-4b54-91bf-fa20b583c990 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:55.779880 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-d66be3a6-449e-4b54-91bf-fa20b583c990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:55.797806 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-b63c1479-a7c9-42d4-989f-fd25dac1d540 admin admin] The token cache is being invalidate because project 59d9161e0aa448c6b737de1a10c41e4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:50:55.798098 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-b63c1479-a7c9-42d4-989f-fd25dac1d540 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:55.799252 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-b63c1479-a7c9-42d4-989f-fd25dac1d540 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:55.800772 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1123/2223] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:50:55 2022] DELETE /identity/v3/projects/59d9161e0aa448c6b737de1a10c41e4e => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:55.805296 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c1479-a7c9-42d4-989f-fd25dac1d540 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:55.806768 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-b63c1479-a7c9-42d4-989f-fd25dac1d540 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:55.837519 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:55.842899 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:55.853576 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1101/2224] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:50:55.861784 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f87bdf-8727-4a82-9f3c-7b8f651fc175 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:55.862111 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f87bdf-8727-4a82-9f3c-7b8f651fc175 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:55.862419 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f87bdf-8727-4a82-9f3c-7b8f651fc175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:55.879971 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c1479-a7c9-42d4-989f-fd25dac1d540 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:55.885718 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27446b5-87d3-4fa5-9927-6b82f3298947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:55.886708 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27446b5-87d3-4fa5-9927-6b82f3298947 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:55.887572 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27446b5-87d3-4fa5-9927-6b82f3298947 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:55.887572 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27446b5-87d3-4fa5-9927-6b82f3298947 admin admin] PATH_INFO: `/v3/users/f8f0020ced3b4b71b88458f7c883cbf5` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:55.888457 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27446b5-87d3-4fa5-9927-6b82f3298947 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8f0020ced3b4b71b88458f7c883cbf5)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:55.888567 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-36f87bdf-8727-4a82-9f3c-7b8f651fc175 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:55.889790 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-36f87bdf-8727-4a82-9f3c-7b8f651fc175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:55.890429 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27446b5-87d3-4fa5-9927-6b82f3298947 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:55.900870 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1102/2225] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:50:55.905926 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f87bdf-8727-4a82-9f3c-7b8f651fc175 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:55.918635 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f87bdf-8727-4a82-9f3c-7b8f651fc175 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:55.923549 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa7de04-475a-40a0-a1f3-b545dbe9436d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:55.924343 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa7de04-475a-40a0-a1f3-b545dbe9436d admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:55.924467 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa7de04-475a-40a0-a1f3-b545dbe9436d admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:55.924703 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa7de04-475a-40a0-a1f3-b545dbe9436d admin admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:55.925378 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa7de04-475a-40a0-a1f3-b545dbe9436d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:50:55.925673 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa7de04-475a-40a0-a1f3-b545dbe9436d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:55.928027 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa7de04-475a-40a0-a1f3-b545dbe9436d admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:55.949311 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1103/2226] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:50:55.950996 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa7de04-475a-40a0-a1f3-b545dbe9436d admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:55.964031 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa7de04-475a-40a0-a1f3-b545dbe9436d admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:55.969863 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3c99e2-2058-454b-9dc5-42a8d28d2024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:55.970475 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c99e2-2058-454b-9dc5-42a8d28d2024 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:55.970824 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c99e2-2058-454b-9dc5-42a8d28d2024 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:55.971098 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c99e2-2058-454b-9dc5-42a8d28d2024 admin admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:55.971735 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3c99e2-2058-454b-9dc5-42a8d28d2024 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:55.974341 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3c99e2-2058-454b-9dc5-42a8d28d2024 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:56.122609 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1104/2227] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:50:55 2022] POST /identity/v3/projects => generated 385 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:50:56.127838 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3c99e2-2058-454b-9dc5-42a8d28d2024 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:56.168197 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d27446b5-87d3-4fa5-9927-6b82f3298947 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8f0020ced3b4b71b88458f7c883cbf5'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:56.209700 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3c99e2-2058-454b-9dc5-42a8d28d2024 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:56.215955 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd3a0b-41bf-48cd-b5af-d9603cfe29a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:56.216571 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd3a0b-41bf-48cd-b5af-d9603cfe29a4 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:56.216886 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd3a0b-41bf-48cd-b5af-d9603cfe29a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:56.217162 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd3a0b-41bf-48cd-b5af-d9603cfe29a4 admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:56.217795 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbd3a0b-41bf-48cd-b5af-d9603cfe29a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:56.220905 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbd3a0b-41bf-48cd-b5af-d9603cfe29a4 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:56.252103 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d27446b5-87d3-4fa5-9927-6b82f3298947 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8f0020ced3b4b71b88458f7c883cbf5'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:56.271983 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d27446b5-87d3-4fa5-9927-6b82f3298947 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8f0020ced3b4b71b88458f7c883cbf5'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:56.278427 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d27446b5-87d3-4fa5-9927-6b82f3298947 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8f0020ced3b4b71b88458f7c883cbf5'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:56.279369 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d27446b5-87d3-4fa5-9927-6b82f3298947 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8f0020ced3b4b71b88458f7c883cbf5'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:56.287509 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1124/2228] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:50:55 2022] DELETE /identity/v3/users/f8f0020ced3b4b71b88458f7c883cbf5 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:50:56.376112 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1105/2229] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:50:56 2022] POST /identity/v3/users => generated 293 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:50:56.380329 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27446b5-87d3-4fa5-9927-6b82f3298947 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:56.382158 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-d27446b5-87d3-4fa5-9927-6b82f3298947 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:56.475006 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27446b5-87d3-4fa5-9927-6b82f3298947 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:56.481363 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b6b75-9c9b-45eb-be26-7a8f3d0d8a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:56.481363 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b6b75-9c9b-45eb-be26-7a8f3d0d8a6e admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:56.481363 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b6b75-9c9b-45eb-be26-7a8f3d0d8a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:56.481655 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b6b75-9c9b-45eb-be26-7a8f3d0d8a6e admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:56.482347 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8b6b75-9c9b-45eb-be26-7a8f3d0d8a6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:56.484652 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8b6b75-9c9b-45eb-be26-7a8f3d0d8a6e admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:56.504302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1125/2230] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:50:56.509349 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd3a0b-41bf-48cd-b5af-d9603cfe29a4 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:56.525035 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd3a0b-41bf-48cd-b5af-d9603cfe29a4 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:56.530440 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3473a6c2-1143-44b8-ae6e-760e5cec5dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:56.530440 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3473a6c2-1143-44b8-ae6e-760e5cec5dea admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:56.531250 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3473a6c2-1143-44b8-ae6e-760e5cec5dea admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:56.532771 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3473a6c2-1143-44b8-ae6e-760e5cec5dea admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:56.536835 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3473a6c2-1143-44b8-ae6e-760e5cec5dea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:56.536835 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3473a6c2-1143-44b8-ae6e-760e5cec5dea admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:56.556151 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1106/2231] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:50:56.557147 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b6b75-9c9b-45eb-be26-7a8f3d0d8a6e admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:56.572597 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b6b75-9c9b-45eb-be26-7a8f3d0d8a6e admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:56.578004 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136fb91-857a-4f81-b7cb-c3f24d4098a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:56.578744 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136fb91-857a-4f81-b7cb-c3f24d4098a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:56.579117 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136fb91-857a-4f81-b7cb-c3f24d4098a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:56.579562 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136fb91-857a-4f81-b7cb-c3f24d4098a3 admin admin] PATH_INFO: `/v3/projects/33c5f1bc7dcb478382befb6595afa320/users/ab3dc7f7cc724424916fabf51874dca6/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:56.613169 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7136fb91-857a-4f81-b7cb-c3f24d4098a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33c5f1bc7dcb478382befb6595afa320, user_id=ab3dc7f7cc724424916fabf51874dca6, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:56.618533 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7136fb91-857a-4f81-b7cb-c3f24d4098a3 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:56.732360 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1126/2232] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:50:56 2022] PUT /identity/v3/projects/33c5f1bc7dcb478382befb6595afa320/users/ab3dc7f7cc724424916fabf51874dca6/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:50:56.751355 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb47019-de6f-4923-97be-62703eb9f62b None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:56.751808 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb47019-de6f-4923-97be-62703eb9f62b None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:56.752230 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb47019-de6f-4923-97be-62703eb9f62b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:56.795000 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-bcb47019-de6f-4923-97be-62703eb9f62b None None] MFA Rules not processed for user `ab3dc7f7cc724424916fabf51874dca6`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:56.796238 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-bcb47019-de6f-4923-97be-62703eb9f62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:56.877691 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136fb91-857a-4f81-b7cb-c3f24d4098a3 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:56.884573 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1107/2233] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:50:56.884870 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-7136fb91-857a-4f81-b7cb-c3f24d4098a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:56.887875 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb47019-de6f-4923-97be-62703eb9f62b None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:56.929028 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:56.929953 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:56.972336 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb47019-de6f-4923-97be-62703eb9f62b None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:56.977299 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f76d9-2ae1-48d0-89e7-9167ce62fc56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:56.977939 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f76d9-2ae1-48d0-89e7-9167ce62fc56 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:56.978224 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f76d9-2ae1-48d0-89e7-9167ce62fc56 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:56.978505 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f76d9-2ae1-48d0-89e7-9167ce62fc56 admin admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:56.979148 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2f76d9-2ae1-48d0-89e7-9167ce62fc56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:56.981118 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2f76d9-2ae1-48d0-89e7-9167ce62fc56 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:56.987537 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136fb91-857a-4f81-b7cb-c3f24d4098a3 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:56.996931 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7bb5e0-a9d1-46dc-b5e2-3539e4cf1457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:56.997626 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7bb5e0-a9d1-46dc-b5e2-3539e4cf1457 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:56.998400 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7bb5e0-a9d1-46dc-b5e2-3539e4cf1457 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:56.998515 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7bb5e0-a9d1-46dc-b5e2-3539e4cf1457 admin admin] PATH_INFO: `/v3/projects/54c6cd633f104c75b4db739ef5b710e5` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:56.999690 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7bb5e0-a9d1-46dc-b5e2-3539e4cf1457 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54c6cd633f104c75b4db739ef5b710e5)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:57.001950 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7bb5e0-a9d1-46dc-b5e2-3539e4cf1457 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:57.121848 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1108/2234] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:50:56 2022] POST /identity/v3/projects => generated 385 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:50:57.123505 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f76d9-2ae1-48d0-89e7-9167ce62fc56 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:57.140951 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-bd7bb5e0-a9d1-46dc-b5e2-3539e4cf1457 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54c6cd633f104c75b4db739ef5b710e5', 'request_id': 'req-bd7bb5e0-a9d1-46dc-b5e2-3539e4cf1457'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:57.148839 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-bd7bb5e0-a9d1-46dc-b5e2-3539e4cf1457 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54c6cd633f104c75b4db739ef5b710e5', 'request_id': 'req-bd7bb5e0-a9d1-46dc-b5e2-3539e4cf1457'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:57.155645 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-bd7bb5e0-a9d1-46dc-b5e2-3539e4cf1457 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54c6cd633f104c75b4db739ef5b710e5', 'request_id': 'req-bd7bb5e0-a9d1-46dc-b5e2-3539e4cf1457'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:57.191248 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f76d9-2ae1-48d0-89e7-9167ce62fc56 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:57.236636 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-bd7bb5e0-a9d1-46dc-b5e2-3539e4cf1457 admin admin] The token cache is being invalidate because project 54c6cd633f104c75b4db739ef5b710e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:50:57.237200 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-bd7bb5e0-a9d1-46dc-b5e2-3539e4cf1457 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:57.240844 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-bd7bb5e0-a9d1-46dc-b5e2-3539e4cf1457 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:50:57.242892 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1127/2235] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:50:56 2022] DELETE /identity/v3/projects/54c6cd633f104c75b4db739ef5b710e5 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:50:57.247837 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7fdd08-f499-4230-8d63-8e602a2e9494 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:57.249276 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7fdd08-f499-4230-8d63-8e602a2e9494 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:57.249820 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7fdd08-f499-4230-8d63-8e602a2e9494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:57.259178 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275a4978-dfd4-40b2-af5b-493c18fde7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:57.260045 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a4978-dfd4-40b2-af5b-493c18fde7c8 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:57.262854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a4978-dfd4-40b2-af5b-493c18fde7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:57.262854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a4978-dfd4-40b2-af5b-493c18fde7c8 admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:57.262854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275a4978-dfd4-40b2-af5b-493c18fde7c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:57.263350 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275a4978-dfd4-40b2-af5b-493c18fde7c8 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:57.289368 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-4b7fdd08-f499-4230-8d63-8e602a2e9494 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:57.290952 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-4b7fdd08-f499-4230-8d63-8e602a2e9494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:57.299943 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1128/2236] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:50:57.308497 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72694c32-acfe-4b35-bf7b-c1716166d591 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:57.308879 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72694c32-acfe-4b35-bf7b-c1716166d591 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:57.309230 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72694c32-acfe-4b35-bf7b-c1716166d591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:57.333891 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-72694c32-acfe-4b35-bf7b-c1716166d591 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:57.335133 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-72694c32-acfe-4b35-bf7b-c1716166d591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:57.345491 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1129/2237] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:50:57.350363 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72694c32-acfe-4b35-bf7b-c1716166d591 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:57.361064 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72694c32-acfe-4b35-bf7b-c1716166d591 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:57.372211 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f6d63f-5f9f-4d0d-9b78-1edd4d13e32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:57.372211 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f6d63f-5f9f-4d0d-9b78-1edd4d13e32a admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:57.372211 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f6d63f-5f9f-4d0d-9b78-1edd4d13e32a admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:57.372211 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f6d63f-5f9f-4d0d-9b78-1edd4d13e32a admin admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:57.372702 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f6d63f-5f9f-4d0d-9b78-1edd4d13e32a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:50:57.372702 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f6d63f-5f9f-4d0d-9b78-1edd4d13e32a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:57.372702 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f6d63f-5f9f-4d0d-9b78-1edd4d13e32a admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:57.388552 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1130/2238] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:50:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:50:57.389469 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f6d63f-5f9f-4d0d-9b78-1edd4d13e32a admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:57.402595 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1109/2239] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:50:57 2022] POST /identity/v3/users => generated 293 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:50:57.408208 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275a4978-dfd4-40b2-af5b-493c18fde7c8 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:57.409946 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-275a4978-dfd4-40b2-af5b-493c18fde7c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:57.422269 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f6d63f-5f9f-4d0d-9b78-1edd4d13e32a admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:57.424045 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275a4978-dfd4-40b2-af5b-493c18fde7c8 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:57.428784 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb86c31-fdf6-4be8-aac3-3e3b5d5eb580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:57.429413 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb86c31-fdf6-4be8-aac3-3e3b5d5eb580 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:57.429826 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb86c31-fdf6-4be8-aac3-3e3b5d5eb580 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:57.430081 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb86c31-fdf6-4be8-aac3-3e3b5d5eb580 admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:57.430683 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b1846d-acc0-46bb-a2f0-4111f0bed895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:57.430917 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb86c31-fdf6-4be8-aac3-3e3b5d5eb580 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:57.431405 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b1846d-acc0-46bb-a2f0-4111f0bed895 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:57.431746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b1846d-acc0-46bb-a2f0-4111f0bed895 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:57.432356 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b1846d-acc0-46bb-a2f0-4111f0bed895 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:57.432820 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb86c31-fdf6-4be8-aac3-3e3b5d5eb580 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:57.435809 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b1846d-acc0-46bb-a2f0-4111f0bed895 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:57.435809 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b1846d-acc0-46bb-a2f0-4111f0bed895 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:57.446740 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1110/2240] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:50:57.450817 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b1846d-acc0-46bb-a2f0-4111f0bed895 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:57.462578 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b1846d-acc0-46bb-a2f0-4111f0bed895 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:57.471633 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce1ed20-379e-4364-83b9-980607d3aac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:57.471633 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce1ed20-379e-4364-83b9-980607d3aac2 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:57.471633 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce1ed20-379e-4364-83b9-980607d3aac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:57.471633 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce1ed20-379e-4364-83b9-980607d3aac2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:57.471955 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce1ed20-379e-4364-83b9-980607d3aac2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:57.471955 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce1ed20-379e-4364-83b9-980607d3aac2 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:57.483910 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1111/2241] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:50:57.488008 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce1ed20-379e-4364-83b9-980607d3aac2 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:57.503048 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce1ed20-379e-4364-83b9-980607d3aac2 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:57.511022 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c904bf2-539a-40cd-8542-44308c186da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:57.512800 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c904bf2-539a-40cd-8542-44308c186da8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:57.513381 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c904bf2-539a-40cd-8542-44308c186da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:57.513959 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c904bf2-539a-40cd-8542-44308c186da8 admin admin] PATH_INFO: `/v3/projects/a4d75c1b48e94d58bbacac8c9356c5a4/users/88f9923a2dda45ada1a6bce88b22aef6/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:57.538459 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c904bf2-539a-40cd-8542-44308c186da8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4d75c1b48e94d58bbacac8c9356c5a4, user_id=88f9923a2dda45ada1a6bce88b22aef6, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:57.545051 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1131/2242] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:50:57 2022] POST /identity/v3/projects => generated 373 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:50:57.546343 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb86c31-fdf6-4be8-aac3-3e3b5d5eb580 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:57.546810 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c904bf2-539a-40cd-8542-44308c186da8 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:57.622147 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:57.628879 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb86c31-fdf6-4be8-aac3-3e3b5d5eb580 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:57.634534 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cb59b8-dbfa-4ec0-ad2e-8db5f0171aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:57.635144 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb59b8-dbfa-4ec0-ad2e-8db5f0171aef admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:57.635427 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb59b8-dbfa-4ec0-ad2e-8db5f0171aef admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:57.635697 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb59b8-dbfa-4ec0-ad2e-8db5f0171aef admin admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:57.636334 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cb59b8-dbfa-4ec0-ad2e-8db5f0171aef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:57.638459 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cb59b8-dbfa-4ec0-ad2e-8db5f0171aef admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:57.654648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1112/2243] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:50:57 2022] PUT /identity/v3/projects/a4d75c1b48e94d58bbacac8c9356c5a4/users/88f9923a2dda45ada1a6bce88b22aef6/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:57.665906 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea3c83-618b-4167-93f5-7a0a635139fa None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:57.666198 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea3c83-618b-4167-93f5-7a0a635139fa None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:57.666475 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea3c83-618b-4167-93f5-7a0a635139fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:57.673646 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:57.704294 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-91ea3c83-618b-4167-93f5-7a0a635139fa None None] MFA Rules not processed for user `88f9923a2dda45ada1a6bce88b22aef6`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:57.704294 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-91ea3c83-618b-4167-93f5-7a0a635139fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:57.774594 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1113/2244] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:50:57.779381 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea3c83-618b-4167-93f5-7a0a635139fa None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:57.819683 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1132/2245] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:50:57 2022] POST /identity/v3/users => generated 287 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:50:57.823945 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cb59b8-dbfa-4ec0-ad2e-8db5f0171aef admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:57.858440 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cb59b8-dbfa-4ec0-ad2e-8db5f0171aef admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:57.873707 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea3c83-618b-4167-93f5-7a0a635139fa None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:57.878468 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c1f071-00fb-428f-8dad-ae76113f1f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:57.879845 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1f071-00fb-428f-8dad-ae76113f1f4e admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:57.880280 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099c7f4-159e-4c59-9235-6f6bd55dffcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:57.881156 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1f071-00fb-428f-8dad-ae76113f1f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:57.881755 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1f071-00fb-428f-8dad-ae76113f1f4e admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:57.882123 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099c7f4-159e-4c59-9235-6f6bd55dffcf admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:57.882399 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099c7f4-159e-4c59-9235-6f6bd55dffcf admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:57.882861 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c1f071-00fb-428f-8dad-ae76113f1f4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:57.883300 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099c7f4-159e-4c59-9235-6f6bd55dffcf admin admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:57.883992 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f099c7f4-159e-4c59-9235-6f6bd55dffcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:57.885932 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c1f071-00fb-428f-8dad-ae76113f1f4e admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:57.886497 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f099c7f4-159e-4c59-9235-6f6bd55dffcf admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:57.902354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1133/2246] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:57 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:50:57.906461 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c1f071-00fb-428f-8dad-ae76113f1f4e admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:57.917098 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c1f071-00fb-428f-8dad-ae76113f1f4e admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:57.922137 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ad348-b596-4e95-9343-ce27190663b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:57.922756 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ad348-b596-4e95-9343-ce27190663b0 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:57.923044 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ad348-b596-4e95-9343-ce27190663b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:57.923321 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ad348-b596-4e95-9343-ce27190663b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:57.924021 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ad348-b596-4e95-9343-ce27190663b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:57.925659 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ad348-b596-4e95-9343-ce27190663b0 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:57.940289 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1134/2247] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:50:57.944615 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ad348-b596-4e95-9343-ce27190663b0 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:57.960374 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ad348-b596-4e95-9343-ce27190663b0 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:57.986697 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79c3e77-8aa9-4b9b-b81c-f5e335a59d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:57.987988 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c3e77-8aa9-4b9b-b81c-f5e335a59d12 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:57.988276 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c3e77-8aa9-4b9b-b81c-f5e335a59d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:57.988695 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c3e77-8aa9-4b9b-b81c-f5e335a59d12 admin admin] PATH_INFO: `/v3/projects/50a332bdc684445cb6d1c937dcee5487/users/80771602ae7f4e27b9a4d4e9d524e37f/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:58.013705 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79c3e77-8aa9-4b9b-b81c-f5e335a59d12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50a332bdc684445cb6d1c937dcee5487, user_id=80771602ae7f4e27b9a4d4e9d524e37f, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:58.016551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79c3e77-8aa9-4b9b-b81c-f5e335a59d12 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:58.024235 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1114/2248] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:50:57 2022] POST /identity/v3/projects => generated 385 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:50:58.028286 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099c7f4-159e-4c59-9235-6f6bd55dffcf admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:58.091796 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099c7f4-159e-4c59-9235-6f6bd55dffcf admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:58.101653 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c7581d-51b9-494c-b8c5-a7c784737524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:58.102497 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c7581d-51b9-494c-b8c5-a7c784737524 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:58.102876 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c7581d-51b9-494c-b8c5-a7c784737524 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:58.103194 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c7581d-51b9-494c-b8c5-a7c784737524 admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:58.103888 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c7581d-51b9-494c-b8c5-a7c784737524 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:58.105904 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c7581d-51b9-494c-b8c5-a7c784737524 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:58.117745 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1135/2249] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:50:57 2022] PUT /identity/v3/projects/50a332bdc684445cb6d1c937dcee5487/users/80771602ae7f4e27b9a4d4e9d524e37f/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:50:58.124942 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f05c92-b85b-4287-b7b7-fbc8f3b24edc None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:58.125580 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f05c92-b85b-4287-b7b7-fbc8f3b24edc None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:58.125580 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f05c92-b85b-4287-b7b7-fbc8f3b24edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:58.150051 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-a4f05c92-b85b-4287-b7b7-fbc8f3b24edc None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:58.151496 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-a4f05c92-b85b-4287-b7b7-fbc8f3b24edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:58.186796 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1115/2250] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:50:58 2022] POST /identity/v3/users => generated 299 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:50:58.200030 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c7581d-51b9-494c-b8c5-a7c784737524 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:58.211847 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c7581d-51b9-494c-b8c5-a7c784737524 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:58.214721 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1136/2251] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:50:58.217100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1862864-51c4-4a25-ae3f-facbd881849f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:58.217982 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1862864-51c4-4a25-ae3f-facbd881849f admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:58.218190 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1862864-51c4-4a25-ae3f-facbd881849f admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:58.218387 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1862864-51c4-4a25-ae3f-facbd881849f admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:58.219090 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1862864-51c4-4a25-ae3f-facbd881849f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:58.220697 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1862864-51c4-4a25-ae3f-facbd881849f admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:58.222702 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:58.225925 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f05c92-b85b-4287-b7b7-fbc8f3b24edc None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:58.229526 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-a4f05c92-b85b-4287-b7b7-fbc8f3b24edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:58.236676 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1116/2252] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:50:58.238121 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1862864-51c4-4a25-ae3f-facbd881849f admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:58.248700 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1862864-51c4-4a25-ae3f-facbd881849f admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:58.253440 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e339a190-c4cb-4289-89cc-79b0289d8570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:58.254089 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e339a190-c4cb-4289-89cc-79b0289d8570 admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:58.254373 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e339a190-c4cb-4289-89cc-79b0289d8570 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:58.255310 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e339a190-c4cb-4289-89cc-79b0289d8570 admin admin] PATH_INFO: `/v3/projects/548f1efb15d2440c82779e412f62fc50/users/9d143d51e006479e9badfd81886ada81/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:58.272039 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e339a190-c4cb-4289-89cc-79b0289d8570 admin admin] RBAC: Authorizing `identity:create_grant(project_id=548f1efb15d2440c82779e412f62fc50, user_id=9d143d51e006479e9badfd81886ada81, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:58.274872 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e339a190-c4cb-4289-89cc-79b0289d8570 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:58.297476 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f05c92-b85b-4287-b7b7-fbc8f3b24edc None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:58.301252 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c20dbd-937b-45c6-829b-b0bb9399ff5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:58.302169 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c20dbd-937b-45c6-829b-b0bb9399ff5c service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:58.302457 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c20dbd-937b-45c6-829b-b0bb9399ff5c service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:58.302737 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c20dbd-937b-45c6-829b-b0bb9399ff5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:58.316395 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c20dbd-937b-45c6-829b-b0bb9399ff5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:58.318475 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c20dbd-937b-45c6-829b-b0bb9399ff5c service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:58.329563 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1137/2253] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:50:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:50:58.369134 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1117/2254] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:50:58 2022] PUT /identity/v3/projects/548f1efb15d2440c82779e412f62fc50/users/9d143d51e006479e9badfd81886ada81/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:50:58.372405 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c20dbd-937b-45c6-829b-b0bb9399ff5c service neutron] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:58.383692 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c20dbd-937b-45c6-829b-b0bb9399ff5c service neutron] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:58.443723 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:58.462537 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14811ec-0174-4951-9e91-4c63fd647b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:58.463229 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14811ec-0174-4951-9e91-4c63fd647b85 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:58.463576 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14811ec-0174-4951-9e91-4c63fd647b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:58.463997 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14811ec-0174-4951-9e91-4c63fd647b85 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:58.464963 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14811ec-0174-4951-9e91-4c63fd647b85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:58.466344 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14811ec-0174-4951-9e91-4c63fd647b85 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:58.479524 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1138/2255] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:50:58 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:50:58.483763 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e339a190-c4cb-4289-89cc-79b0289d8570 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:58.497432 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e339a190-c4cb-4289-89cc-79b0289d8570 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:58.503335 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f82726-9a1a-4a06-bb72-50b0f01cc648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:58.513454 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f82726-9a1a-4a06-bb72-50b0f01cc648 admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:58.513454 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f82726-9a1a-4a06-bb72-50b0f01cc648 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:58.513454 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f82726-9a1a-4a06-bb72-50b0f01cc648 admin admin] PATH_INFO: `/v3/projects/548f1efb15d2440c82779e412f62fc50/users/9d143d51e006479e9badfd81886ada81/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:58.513454 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f82726-9a1a-4a06-bb72-50b0f01cc648 admin admin] RBAC: Authorizing `identity:create_grant(project_id=548f1efb15d2440c82779e412f62fc50, user_id=9d143d51e006479e9badfd81886ada81, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:58.513454 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f82726-9a1a-4a06-bb72-50b0f01cc648 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:58.522876 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1118/2256] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:50:58 2022] PUT /identity/v3/projects/548f1efb15d2440c82779e412f62fc50/users/9d143d51e006479e9badfd81886ada81/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:50:58.532815 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3873acc5-6ac0-4617-b186-6df3186b5a2f None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:58.533107 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3873acc5-6ac0-4617-b186-6df3186b5a2f None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:58.533439 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3873acc5-6ac0-4617-b186-6df3186b5a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:58.575486 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-3873acc5-6ac0-4617-b186-6df3186b5a2f None None] MFA Rules not processed for user `9d143d51e006479e9badfd81886ada81`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:50:58.576696 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-3873acc5-6ac0-4617-b186-6df3186b5a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:58.652884 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1139/2257] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:50:58 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:50:58.666416 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f82726-9a1a-4a06-bb72-50b0f01cc648 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:58.668711 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-c7f82726-9a1a-4a06-bb72-50b0f01cc648 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:58.730147 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f82726-9a1a-4a06-bb72-50b0f01cc648 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:58.733787 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6817e80-e667-4ab7-a1b8-28051b8cd565 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:58.734381 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6817e80-e667-4ab7-a1b8-28051b8cd565 service cinder] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:58.734658 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6817e80-e667-4ab7-a1b8-28051b8cd565 service cinder] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:58.734928 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6817e80-e667-4ab7-a1b8-28051b8cd565 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:58.736670 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-e6817e80-e667-4ab7-a1b8-28051b8cd565 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:58.795928 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6817e80-e667-4ab7-a1b8-28051b8cd565 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:58.797389 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6817e80-e667-4ab7-a1b8-28051b8cd565 service cinder] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:58.807010 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1119/2258] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:50:58 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:51:00.018699 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b088f69-4c53-4da5-8c02-cd6a76b73604 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:00.019223 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b088f69-4c53-4da5-8c02-cd6a76b73604 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:00.019223 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b088f69-4c53-4da5-8c02-cd6a76b73604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:00.063811 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-7b088f69-4c53-4da5-8c02-cd6a76b73604 None None] MFA Rules not processed for user `80771602ae7f4e27b9a4d4e9d524e37f`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:51:00.068923 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-7b088f69-4c53-4da5-8c02-cd6a76b73604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:00.149764 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1140/2259] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:51:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:51:00.154195 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6817e80-e667-4ab7-a1b8-28051b8cd565 service cinder] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:00.234829 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6817e80-e667-4ab7-a1b8-28051b8cd565 service cinder] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:00.240141 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7694e48-bf0e-4730-8921-b3a4a48f4a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:00.240855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7694e48-bf0e-4730-8921-b3a4a48f4a3e admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:00.241207 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7694e48-bf0e-4730-8921-b3a4a48f4a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:00.241484 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7694e48-bf0e-4730-8921-b3a4a48f4a3e admin admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:00.242130 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7694e48-bf0e-4730-8921-b3a4a48f4a3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:00.244031 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7694e48-bf0e-4730-8921-b3a4a48f4a3e admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:00.378982 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:51:00.389385 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1120/2260] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:51:00 2022] POST /identity/v3/projects => generated 373 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:51:00.393309 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b088f69-4c53-4da5-8c02-cd6a76b73604 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:00.405283 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b088f69-4c53-4da5-8c02-cd6a76b73604 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:00.461157 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:51:00.469547 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ea59fd-6500-4434-be36-b4dc84242583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:00.470276 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea59fd-6500-4434-be36-b4dc84242583 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:00.470557 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea59fd-6500-4434-be36-b4dc84242583 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:00.471027 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea59fd-6500-4434-be36-b4dc84242583 admin admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:00.471700 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ea59fd-6500-4434-be36-b4dc84242583 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:00.474475 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ea59fd-6500-4434-be36-b4dc84242583 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:00.603539 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1141/2261] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:51:00 2022] POST /identity/v3/users => generated 293 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:51:00.609141 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7694e48-bf0e-4730-8921-b3a4a48f4a3e admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:00.630547 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7694e48-bf0e-4730-8921-b3a4a48f4a3e admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:00.636502 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a8f881-d254-4e96-80f5-66e9dd289da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:00.637185 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8f881-d254-4e96-80f5-66e9dd289da3 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:00.637489 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8f881-d254-4e96-80f5-66e9dd289da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:00.637825 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8f881-d254-4e96-80f5-66e9dd289da3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:00.638411 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a8f881-d254-4e96-80f5-66e9dd289da3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:00.640079 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a8f881-d254-4e96-80f5-66e9dd289da3 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:00.652777 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1121/2262] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:51:00 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:51:00.661501 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ea59fd-6500-4434-be36-b4dc84242583 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:00.672089 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ea59fd-6500-4434-be36-b4dc84242583 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:00.677961 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b328d7-a9a4-46fc-8c82-6f51c8c36c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:00.678771 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b328d7-a9a4-46fc-8c82-6f51c8c36c66 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:00.679132 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b328d7-a9a4-46fc-8c82-6f51c8c36c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:00.679411 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b328d7-a9a4-46fc-8c82-6f51c8c36c66 admin admin] PATH_INFO: `/v3/projects/49c61540797b4ecaa717cfa6d884c01f/users/3b581d01552a40dda1838870c8b8e2ce/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:00.697558 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b328d7-a9a4-46fc-8c82-6f51c8c36c66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49c61540797b4ecaa717cfa6d884c01f, user_id=3b581d01552a40dda1838870c8b8e2ce, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:00.700293 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b328d7-a9a4-46fc-8c82-6f51c8c36c66 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:00.727640 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b55b3-2615-4db9-9aed-394bcaf1b216 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:00.727640 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b55b3-2615-4db9-9aed-394bcaf1b216 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:00.727640 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b55b3-2615-4db9-9aed-394bcaf1b216 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:00.728892 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1122/2263] 10.222.0.52 () {54 vars in 921 bytes} [Mon Aug 29 22:51:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:51:00.734900 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad3519-9e17-424c-8a44-56ee74be2d2d None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:00.735184 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad3519-9e17-424c-8a44-56ee74be2d2d None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:00.735458 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad3519-9e17-424c-8a44-56ee74be2d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:00.756785 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-eaad3519-9e17-424c-8a44-56ee74be2d2d None None] MFA Rules not processed for user `bdbf961e02934e88af3a72d6218f32fa`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:51:00.757992 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-eaad3519-9e17-424c-8a44-56ee74be2d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:00.825561 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1123/2264] 10.222.0.52 () {58 vars in 1018 bytes} [Mon Aug 29 22:51:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:51:00.837368 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad3519-9e17-424c-8a44-56ee74be2d2d None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:00.838708 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-eaad3519-9e17-424c-8a44-56ee74be2d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:00.849655 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1142/2265] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:51:00 2022] PUT /identity/v3/projects/49c61540797b4ecaa717cfa6d884c01f/users/3b581d01552a40dda1838870c8b8e2ce/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:51:00.853667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b328d7-a9a4-46fc-8c82-6f51c8c36c66 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:00.902823 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad3519-9e17-424c-8a44-56ee74be2d2d None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:00.930854 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b328d7-a9a4-46fc-8c82-6f51c8c36c66 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:00.935915 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73664d2-920d-4715-9350-f8ef3eec127b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:00.936417 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73664d2-920d-4715-9350-f8ef3eec127b admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:00.936690 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73664d2-920d-4715-9350-f8ef3eec127b admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:00.937199 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73664d2-920d-4715-9350-f8ef3eec127b admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:00.937863 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73664d2-920d-4715-9350-f8ef3eec127b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:00.939457 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73664d2-920d-4715-9350-f8ef3eec127b admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:00.953315 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1143/2266] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:51:00 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:51:00.958422 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73664d2-920d-4715-9350-f8ef3eec127b admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:00.962688 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd790b-88d4-4cb0-9b12-d7b8b445792f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c2569e7cd774199aa5750fd6a850060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:00.963209 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd790b-88d4-4cb0-9b12-d7b8b445792f service glance] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:00.963491 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd790b-88d4-4cb0-9b12-d7b8b445792f service glance] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:00.965146 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd790b-88d4-4cb0-9b12-d7b8b445792f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:00.980831 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73664d2-920d-4715-9350-f8ef3eec127b admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:00.980831 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be89e903-f0f0-4b89-931f-7ae8865f9655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:00.980831 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89e903-f0f0-4b89-931f-7ae8865f9655 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:00.980831 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89e903-f0f0-4b89-931f-7ae8865f9655 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:00.981237 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89e903-f0f0-4b89-931f-7ae8865f9655 admin admin] PATH_INFO: `/v3/projects/49c61540797b4ecaa717cfa6d884c01f/users/3b581d01552a40dda1838870c8b8e2ce/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:00.981237 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be89e903-f0f0-4b89-931f-7ae8865f9655 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49c61540797b4ecaa717cfa6d884c01f, user_id=3b581d01552a40dda1838870c8b8e2ce, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:00.981237 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be89e903-f0f0-4b89-931f-7ae8865f9655 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:00.993892 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1144/2267] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:51:00 2022] PUT /identity/v3/projects/49c61540797b4ecaa717cfa6d884c01f/users/3b581d01552a40dda1838870c8b8e2ce/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:51:01.023802 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:51:01.048366 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbd790b-88d4-4cb0-9b12-d7b8b445792f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:01.049878 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbd790b-88d4-4cb0-9b12-d7b8b445792f service glance] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:01.061987 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1124/2268] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:51:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:51:01.306219 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a11a4-dba3-4e8b-86c0-3bed8e9425af None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:01.306753 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a11a4-dba3-4e8b-86c0-3bed8e9425af None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:01.306873 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a11a4-dba3-4e8b-86c0-3bed8e9425af None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:01.308193 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1145/2269] 10.222.0.52 () {54 vars in 921 bytes} [Mon Aug 29 22:51:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:51:01.314126 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bacb226-ac08-41b5-87bf-90c27a518d0e None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:01.314585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bacb226-ac08-41b5-87bf-90c27a518d0e None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:01.315182 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bacb226-ac08-41b5-87bf-90c27a518d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:01.339167 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-6bacb226-ac08-41b5-87bf-90c27a518d0e None None] MFA Rules not processed for user `0fe5e8c4cb324ffb9d5b80f18d123727`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:51:01.340375 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-6bacb226-ac08-41b5-87bf-90c27a518d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:01.419714 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1125/2270] 10.222.0.52 () {58 vars in 1018 bytes} [Mon Aug 29 22:51:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:51:01.435079 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730a11a4-dba3-4e8b-86c0-3bed8e9425af None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:01.459996 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730a11a4-dba3-4e8b-86c0-3bed8e9425af None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:01.464015 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdb3de7-dfd7-406f-8c24-4e9d628fc5a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:01.465028 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb3de7-dfd7-406f-8c24-4e9d628fc5a9 service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:01.465028 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb3de7-dfd7-406f-8c24-4e9d628fc5a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:01.465174 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb3de7-dfd7-406f-8c24-4e9d628fc5a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:01.475747 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdb3de7-dfd7-406f-8c24-4e9d628fc5a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:01.477909 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdb3de7-dfd7-406f-8c24-4e9d628fc5a9 service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:01.486110 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bacb226-ac08-41b5-87bf-90c27a518d0e None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:01.487743 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-6bacb226-ac08-41b5-87bf-90c27a518d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:01.491066 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1146/2271] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:51:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:51:01.575974 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bacb226-ac08-41b5-87bf-90c27a518d0e None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:01.591548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18625e88-f962-4199-b8b3-ca24526664e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:01.591548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18625e88-f962-4199-b8b3-ca24526664e9 service cinder] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:01.591548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18625e88-f962-4199-b8b3-ca24526664e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:01.591548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18625e88-f962-4199-b8b3-ca24526664e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:01.597646 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a907b-66d0-4b03-823f-ecd837268e30 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:01.597935 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a907b-66d0-4b03-823f-ecd837268e30 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:01.598208 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a907b-66d0-4b03-823f-ecd837268e30 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:01.600614 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1147/2272] 10.222.0.52 () {54 vars in 921 bytes} [Mon Aug 29 22:51:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:51:01.607281 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f764f5e9-6b68-49c2-b348-703562dccd9c None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:01.607570 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f764f5e9-6b68-49c2-b348-703562dccd9c None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:01.607849 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f764f5e9-6b68-49c2-b348-703562dccd9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:01.631769 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-f764f5e9-6b68-49c2-b348-703562dccd9c None None] MFA Rules not processed for user `bdbf961e02934e88af3a72d6218f32fa`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:51:01.633022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-f764f5e9-6b68-49c2-b348-703562dccd9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:01.663543 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18625e88-f962-4199-b8b3-ca24526664e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:01.668464 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18625e88-f962-4199-b8b3-ca24526664e9 service cinder] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:01.697677 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1126/2273] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:51:01 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:51:01.768621 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1148/2274] 10.222.0.52 () {58 vars in 1018 bytes} [Mon Aug 29 22:51:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:51:01.783069 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18625e88-f962-4199-b8b3-ca24526664e9 service cinder] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:01.784929 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-18625e88-f962-4199-b8b3-ca24526664e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:01.800145 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18625e88-f962-4199-b8b3-ca24526664e9 service cinder] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:01.805659 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f23d37-856d-4004-8394-7adaa79779c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:01.806207 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f23d37-856d-4004-8394-7adaa79779c5 service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:01.806502 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f23d37-856d-4004-8394-7adaa79779c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:01.806777 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f23d37-856d-4004-8394-7adaa79779c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:01.820762 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f23d37-856d-4004-8394-7adaa79779c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:01.822407 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f23d37-856d-4004-8394-7adaa79779c5 service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:01.837044 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1127/2275] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:51:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:51:02.942897 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9ca38-66ac-495a-977e-4e2a227c1ff0 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:02.943450 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9ca38-66ac-495a-977e-4e2a227c1ff0 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:02.943450 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9ca38-66ac-495a-977e-4e2a227c1ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:02.975465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-daf9ca38-66ac-495a-977e-4e2a227c1ff0 None None] MFA Rules not processed for user `3b581d01552a40dda1838870c8b8e2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:51:02.976883 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-daf9ca38-66ac-495a-977e-4e2a227c1ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:02.990192 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:51:03.068284 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1149/2276] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:51:02 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:51:03.079735 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f23d37-856d-4004-8394-7adaa79779c5 service neutron] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:03.090445 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f23d37-856d-4004-8394-7adaa79779c5 service neutron] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:03.094073 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc74874-1c04-4065-a8a2-edfb1361da55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:03.094678 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc74874-1c04-4065-a8a2-edfb1361da55 service cinder] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:03.094955 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc74874-1c04-4065-a8a2-edfb1361da55 service cinder] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:03.095231 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc74874-1c04-4065-a8a2-edfb1361da55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:03.121711 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:51:03.155435 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc74874-1c04-4065-a8a2-edfb1361da55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:03.156896 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc74874-1c04-4065-a8a2-edfb1361da55 service cinder] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:03.167586 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1128/2277] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:51:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:51:03.731272 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf9ca38-66ac-495a-977e-4e2a227c1ff0 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:03.741555 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf9ca38-66ac-495a-977e-4e2a227c1ff0 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:03.745169 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd63f802-b303-44cf-8ec6-c0dac9aa99a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:03.745770 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63f802-b303-44cf-8ec6-c0dac9aa99a2 service cinder] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:03.746050 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63f802-b303-44cf-8ec6-c0dac9aa99a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:03.746319 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63f802-b303-44cf-8ec6-c0dac9aa99a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:03.807854 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd63f802-b303-44cf-8ec6-c0dac9aa99a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:03.809297 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd63f802-b303-44cf-8ec6-c0dac9aa99a2 service cinder] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:03.819723 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1150/2278] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:51:03 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:51:07.160691 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc74874-1c04-4065-a8a2-edfb1361da55 service cinder] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:07.170283 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc74874-1c04-4065-a8a2-edfb1361da55 service cinder] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:07.174449 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a82628-1634-4722-b268-46f22dff0cf6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:07.175043 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a82628-1634-4722-b268-46f22dff0cf6 service cinder] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:07.175318 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a82628-1634-4722-b268-46f22dff0cf6 service cinder] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:07.175588 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a82628-1634-4722-b268-46f22dff0cf6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:07.185155 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a82628-1634-4722-b268-46f22dff0cf6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:07.186759 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a82628-1634-4722-b268-46f22dff0cf6 service cinder] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:07.198599 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1129/2279] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:51:07 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:51:09.092936 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b4437-b0de-48c2-b994-d96c30196f74 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:09.093427 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b4437-b0de-48c2-b994-d96c30196f74 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:09.093523 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b4437-b0de-48c2-b994-d96c30196f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:09.114373 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-8a0b4437-b0de-48c2-b994-d96c30196f74 None None] MFA Rules not processed for user `9c3c51d98e274150933a1306f7747f42`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:51:09.115583 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-8a0b4437-b0de-48c2-b994-d96c30196f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:09.180251 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1151/2280] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:51:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:51:11.195213 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a82628-1634-4722-b268-46f22dff0cf6 service cinder] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:11.254744 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e4d9f0-d227-48ca-891b-fb72bd2b6f46 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:11.255299 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e4d9f0-d227-48ca-891b-fb72bd2b6f46 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:11.255299 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e4d9f0-d227-48ca-891b-fb72bd2b6f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:11.286671 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-58e4d9f0-d227-48ca-891b-fb72bd2b6f46 None None] MFA Rules not processed for user `f77dfe78b1a8405084c06d9acee96511`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:51:11.293770 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-58e4d9f0-d227-48ca-891b-fb72bd2b6f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:11.332971 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a82628-1634-4722-b268-46f22dff0cf6 service cinder] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:11.344041 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8468a7a5-e214-40e5-b30f-2a96f178d20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:11.344676 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8468a7a5-e214-40e5-b30f-2a96f178d20d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:11.344985 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8468a7a5-e214-40e5-b30f-2a96f178d20d admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:11.345256 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8468a7a5-e214-40e5-b30f-2a96f178d20d admin admin] PATH_INFO: `/v3/users/ab3dc7f7cc724424916fabf51874dca6` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:11.348739 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8468a7a5-e214-40e5-b30f-2a96f178d20d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab3dc7f7cc724424916fabf51874dca6)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:11.349286 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:51:11.351149 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8468a7a5-e214-40e5-b30f-2a96f178d20d admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:11.363757 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1152/2281] 10.222.0.52 () {58 vars in 1009 bytes} [Mon Aug 29 22:51:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:51:11.380619 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e4d9f0-d227-48ca-891b-fb72bd2b6f46 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:11.384190 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-58e4d9f0-d227-48ca-891b-fb72bd2b6f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:11.469441 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e4d9f0-d227-48ca-891b-fb72bd2b6f46 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:11.476152 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215b3b96-defa-4887-bbee-62fe412b2a5a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57c175ac90e545358746176972ff39a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:11.476800 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b3b96-defa-4887-bbee-62fe412b2a5a service swift] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:11.477081 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b3b96-defa-4887-bbee-62fe412b2a5a service swift] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:11.477355 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b3b96-defa-4887-bbee-62fe412b2a5a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:11.486643 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215b3b96-defa-4887-bbee-62fe412b2a5a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:11.488305 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215b3b96-defa-4887-bbee-62fe412b2a5a service swift] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:11.497702 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1153/2282] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Aug 29 22:51:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 22:51:11.672599 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-8468a7a5-e214-40e5-b30f-2a96f178d20d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab3dc7f7cc724424916fabf51874dca6'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:11.689284 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-8468a7a5-e214-40e5-b30f-2a96f178d20d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab3dc7f7cc724424916fabf51874dca6'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:11.698320 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-8468a7a5-e214-40e5-b30f-2a96f178d20d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab3dc7f7cc724424916fabf51874dca6'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:11.832698 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-8468a7a5-e214-40e5-b30f-2a96f178d20d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab3dc7f7cc724424916fabf51874dca6'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:11.833530 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-8468a7a5-e214-40e5-b30f-2a96f178d20d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab3dc7f7cc724424916fabf51874dca6'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:11.841661 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1130/2283] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:51:11 2022] DELETE /identity/v3/users/ab3dc7f7cc724424916fabf51874dca6 => generated 0 bytes in 647 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:51:11.846354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4cb1f-7ce1-4c87-adc2-79a555772d63 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:11.846645 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4cb1f-7ce1-4c87-adc2-79a555772d63 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:11.846961 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4cb1f-7ce1-4c87-adc2-79a555772d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:11.888916 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-57c4cb1f-7ce1-4c87-adc2-79a555772d63 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:51:11.890406 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-57c4cb1f-7ce1-4c87-adc2-79a555772d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:12.014663 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1154/2284] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:51:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:51:12.028345 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8468a7a5-e214-40e5-b30f-2a96f178d20d admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:12.029892 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-8468a7a5-e214-40e5-b30f-2a96f178d20d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:12.098626 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:51:12.126717 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8468a7a5-e214-40e5-b30f-2a96f178d20d admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:12.131769 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b6c580-7260-4f00-8351-4fa9d60f3d6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:12.132629 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6c580-7260-4f00-8351-4fa9d60f3d6f service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:12.133032 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6c580-7260-4f00-8351-4fa9d60f3d6f service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:12.133436 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6c580-7260-4f00-8351-4fa9d60f3d6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:12.144438 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b6c580-7260-4f00-8351-4fa9d60f3d6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:12.145926 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b6c580-7260-4f00-8351-4fa9d60f3d6f service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:12.158717 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1131/2285] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:51:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:51:12.356897 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c4cb1f-7ce1-4c87-adc2-79a555772d63 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:12.358445 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-57c4cb1f-7ce1-4c87-adc2-79a555772d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:12.371406 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c4cb1f-7ce1-4c87-adc2-79a555772d63 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:12.376664 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f106a92-fd3c-43d6-8bab-0aca9ae6f67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:12.377638 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f106a92-fd3c-43d6-8bab-0aca9ae6f67d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:12.378120 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f106a92-fd3c-43d6-8bab-0aca9ae6f67d admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:12.378608 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f106a92-fd3c-43d6-8bab-0aca9ae6f67d admin admin] PATH_INFO: `/v3/users/a45666139f9241898ec015bb83f362af` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:12.380069 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f106a92-fd3c-43d6-8bab-0aca9ae6f67d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a45666139f9241898ec015bb83f362af)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:12.382587 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f106a92-fd3c-43d6-8bab-0aca9ae6f67d admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:12.633497 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-5f106a92-fd3c-43d6-8bab-0aca9ae6f67d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a45666139f9241898ec015bb83f362af'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:12.769996 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-5f106a92-fd3c-43d6-8bab-0aca9ae6f67d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a45666139f9241898ec015bb83f362af'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:12.775042 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b6c580-7260-4f00-8351-4fa9d60f3d6f service neutron] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:12.776541 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-96b6c580-7260-4f00-8351-4fa9d60f3d6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:12.807818 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b6c580-7260-4f00-8351-4fa9d60f3d6f service neutron] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:12.808483 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-5f106a92-fd3c-43d6-8bab-0aca9ae6f67d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a45666139f9241898ec015bb83f362af'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:12.823444 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:51:12.826626 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dab224a-0255-49cb-bb44-d9329b3639a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:12.828636 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-5f106a92-fd3c-43d6-8bab-0aca9ae6f67d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a45666139f9241898ec015bb83f362af'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:12.828959 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab224a-0255-49cb-bb44-d9329b3639a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:12.829451 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-5f106a92-fd3c-43d6-8bab-0aca9ae6f67d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a45666139f9241898ec015bb83f362af'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:12.829670 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab224a-0255-49cb-bb44-d9329b3639a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:12.830152 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab224a-0255-49cb-bb44-d9329b3639a2 admin admin] PATH_INFO: `/v3/projects/33c5f1bc7dcb478382befb6595afa320` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:12.831829 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dab224a-0255-49cb-bb44-d9329b3639a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33c5f1bc7dcb478382befb6595afa320)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:12.835232 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dab224a-0255-49cb-bb44-d9329b3639a2 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:12.841657 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1155/2286] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:51:12 2022] DELETE /identity/v3/users/a45666139f9241898ec015bb83f362af => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:51:12.846760 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d964e74-643d-4d10-80a4-03f392871c94 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:12.847167 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d964e74-643d-4d10-80a4-03f392871c94 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:12.847547 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d964e74-643d-4d10-80a4-03f392871c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:12.900868 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-6d964e74-643d-4d10-80a4-03f392871c94 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:51:12.902549 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-6d964e74-643d-4d10-80a4-03f392871c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:12.974144 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-1dab224a-0255-49cb-bb44-d9329b3639a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33c5f1bc7dcb478382befb6595afa320', 'request_id': 'req-1dab224a-0255-49cb-bb44-d9329b3639a2'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:13.015641 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1156/2287] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:51:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:51:13.037870 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d964e74-643d-4d10-80a4-03f392871c94 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:13.037870 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-6d964e74-643d-4d10-80a4-03f392871c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:13.086527 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-1dab224a-0255-49cb-bb44-d9329b3639a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33c5f1bc7dcb478382befb6595afa320', 'request_id': 'req-1dab224a-0255-49cb-bb44-d9329b3639a2'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:13.094978 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-1dab224a-0255-49cb-bb44-d9329b3639a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33c5f1bc7dcb478382befb6595afa320', 'request_id': 'req-1dab224a-0255-49cb-bb44-d9329b3639a2'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:13.135930 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-1dab224a-0255-49cb-bb44-d9329b3639a2 admin admin] The token cache is being invalidate because project 33c5f1bc7dcb478382befb6595afa320 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:51:13.136599 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-1dab224a-0255-49cb-bb44-d9329b3639a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:13.153611 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-1dab224a-0255-49cb-bb44-d9329b3639a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:13.158219 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1132/2288] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:51:12 2022] DELETE /identity/v3/projects/33c5f1bc7dcb478382befb6595afa320 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:51:13.159375 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dab224a-0255-49cb-bb44-d9329b3639a2 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:13.160823 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-1dab224a-0255-49cb-bb44-d9329b3639a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:13.178282 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d964e74-643d-4d10-80a4-03f392871c94 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:13.307124 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab77fb48-1732-4ad9-8169-79209c40b143 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:13.308064 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77fb48-1732-4ad9-8169-79209c40b143 service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:13.308355 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77fb48-1732-4ad9-8169-79209c40b143 service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:13.308858 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77fb48-1732-4ad9-8169-79209c40b143 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:13.311696 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-ab77fb48-1732-4ad9-8169-79209c40b143 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:13.421767 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dab224a-0255-49cb-bb44-d9329b3639a2 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:13.437398 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab77fb48-1732-4ad9-8169-79209c40b143 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:13.439722 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab77fb48-1732-4ad9-8169-79209c40b143 service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:13.475730 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df9587d-ce9f-4179-bdd8-ec0d1515bcff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:13.478872 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1157/2289] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:51:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:51:13.485278 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9587d-ce9f-4179-bdd8-ec0d1515bcff admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:13.485278 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9587d-ce9f-4179-bdd8-ec0d1515bcff admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:13.493991 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9587d-ce9f-4179-bdd8-ec0d1515bcff admin admin] PATH_INFO: `/v3/users/88f9923a2dda45ada1a6bce88b22aef6` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:13.499222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df9587d-ce9f-4179-bdd8-ec0d1515bcff admin admin] RBAC: Authorizing `identity:delete_user(user_id=88f9923a2dda45ada1a6bce88b22aef6)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:13.507293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df9587d-ce9f-4179-bdd8-ec0d1515bcff admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:13.919481 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-1df9587d-ce9f-4179-bdd8-ec0d1515bcff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88f9923a2dda45ada1a6bce88b22aef6'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:13.939563 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:51:13.949171 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-1df9587d-ce9f-4179-bdd8-ec0d1515bcff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88f9923a2dda45ada1a6bce88b22aef6'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:13.957503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-1df9587d-ce9f-4179-bdd8-ec0d1515bcff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88f9923a2dda45ada1a6bce88b22aef6'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:14.546735 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-1df9587d-ce9f-4179-bdd8-ec0d1515bcff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88f9923a2dda45ada1a6bce88b22aef6'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:14.547725 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-1df9587d-ce9f-4179-bdd8-ec0d1515bcff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88f9923a2dda45ada1a6bce88b22aef6'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:14.567017 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1133/2290] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:51:13 2022] DELETE /identity/v3/users/88f9923a2dda45ada1a6bce88b22aef6 => generated 0 bytes in 1408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:51:15.133339 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab77fb48-1732-4ad9-8169-79209c40b143 service neutron] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:15.134871 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-ab77fb48-1732-4ad9-8169-79209c40b143 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:15.149767 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab77fb48-1732-4ad9-8169-79209c40b143 service neutron] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:15.299407 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad5f51-08e1-4bd8-ad4e-936d3d1a8b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:15.300204 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad5f51-08e1-4bd8-ad4e-936d3d1a8b5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:15.300602 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad5f51-08e1-4bd8-ad4e-936d3d1a8b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:15.301038 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad5f51-08e1-4bd8-ad4e-936d3d1a8b5d admin admin] PATH_INFO: `/v3/projects/90e8bdd751634f0596fcd55fb1b4aa24` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:15.302309 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfad5f51-08e1-4bd8-ad4e-936d3d1a8b5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=90e8bdd751634f0596fcd55fb1b4aa24)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:15.305155 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfad5f51-08e1-4bd8-ad4e-936d3d1a8b5d admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:15.437304 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-bfad5f51-08e1-4bd8-ad4e-936d3d1a8b5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90e8bdd751634f0596fcd55fb1b4aa24', 'request_id': 'req-bfad5f51-08e1-4bd8-ad4e-936d3d1a8b5d'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:15.443421 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-bfad5f51-08e1-4bd8-ad4e-936d3d1a8b5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90e8bdd751634f0596fcd55fb1b4aa24', 'request_id': 'req-bfad5f51-08e1-4bd8-ad4e-936d3d1a8b5d'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:15.452414 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-bfad5f51-08e1-4bd8-ad4e-936d3d1a8b5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90e8bdd751634f0596fcd55fb1b4aa24', 'request_id': 'req-bfad5f51-08e1-4bd8-ad4e-936d3d1a8b5d'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:15.514369 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df9587d-ce9f-4179-bdd8-ec0d1515bcff admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:15.516318 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-1df9587d-ce9f-4179-bdd8-ec0d1515bcff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:15.539764 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:51:15.552605 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-bfad5f51-08e1-4bd8-ad4e-936d3d1a8b5d admin admin] The token cache is being invalidate because project 90e8bdd751634f0596fcd55fb1b4aa24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:51:15.556660 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-bfad5f51-08e1-4bd8-ad4e-936d3d1a8b5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:15.559184 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-bfad5f51-08e1-4bd8-ad4e-936d3d1a8b5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:15.562130 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1158/2291] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:51:15 2022] DELETE /identity/v3/projects/90e8bdd751634f0596fcd55fb1b4aa24 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:51:15.571786 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456d6e8-ae1b-4d72-94a3-b31c18d340db None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:15.571786 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456d6e8-ae1b-4d72-94a3-b31c18d340db None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:15.571786 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456d6e8-ae1b-4d72-94a3-b31c18d340db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:15.581607 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df9587d-ce9f-4179-bdd8-ec0d1515bcff admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:15.595615 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-3456d6e8-ae1b-4d72-94a3-b31c18d340db None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:51:15.597082 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-3456d6e8-ae1b-4d72-94a3-b31c18d340db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:15.700960 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac633f7-49e6-4e23-8fab-bf4dbc4fd74c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:15.702037 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac633f7-49e6-4e23-8fab-bf4dbc4fd74c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:15.702037 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac633f7-49e6-4e23-8fab-bf4dbc4fd74c admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:15.702223 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac633f7-49e6-4e23-8fab-bf4dbc4fd74c admin admin] PATH_INFO: `/v3/projects/a4d75c1b48e94d58bbacac8c9356c5a4` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:15.703783 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac633f7-49e6-4e23-8fab-bf4dbc4fd74c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4d75c1b48e94d58bbacac8c9356c5a4)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:15.709423 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac633f7-49e6-4e23-8fab-bf4dbc4fd74c admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:15.721895 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1159/2292] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:51:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:51:15.731948 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f40c72-3bba-46ee-838e-aa5f326df4cb None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:15.731948 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f40c72-3bba-46ee-838e-aa5f326df4cb None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:15.731948 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f40c72-3bba-46ee-838e-aa5f326df4cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:15.789139 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-a2f40c72-3bba-46ee-838e-aa5f326df4cb None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:51:15.794425 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-a2f40c72-3bba-46ee-838e-aa5f326df4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:15.801541 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1160/2293] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:51:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:51:15.805895 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f40c72-3bba-46ee-838e-aa5f326df4cb None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:15.815643 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f40c72-3bba-46ee-838e-aa5f326df4cb None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:15.821504 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085487ea-fb65-4d12-818f-dc7710893083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:15.822127 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085487ea-fb65-4d12-818f-dc7710893083 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:15.822413 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085487ea-fb65-4d12-818f-dc7710893083 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:15.822686 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085487ea-fb65-4d12-818f-dc7710893083 admin admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:15.823341 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085487ea-fb65-4d12-818f-dc7710893083 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:51:15.823649 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085487ea-fb65-4d12-818f-dc7710893083 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:15.825746 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085487ea-fb65-4d12-818f-dc7710893083 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:15.842674 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1161/2294] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:51:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:51:15.848627 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085487ea-fb65-4d12-818f-dc7710893083 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:15.855861 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-0ac633f7-49e6-4e23-8fab-bf4dbc4fd74c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4d75c1b48e94d58bbacac8c9356c5a4', 'request_id': 'req-0ac633f7-49e6-4e23-8fab-bf4dbc4fd74c'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:15.860652 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085487ea-fb65-4d12-818f-dc7710893083 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:15.966047 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc004570-a42c-4a62-ac67-7c0f7f545d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:15.966672 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc004570-a42c-4a62-ac67-7c0f7f545d4e admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:15.966959 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc004570-a42c-4a62-ac67-7c0f7f545d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:15.967232 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc004570-a42c-4a62-ac67-7c0f7f545d4e admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:15.967883 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc004570-a42c-4a62-ac67-7c0f7f545d4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:15.973178 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc004570-a42c-4a62-ac67-7c0f7f545d4e admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:16.149673 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-0ac633f7-49e6-4e23-8fab-bf4dbc4fd74c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4d75c1b48e94d58bbacac8c9356c5a4', 'request_id': 'req-0ac633f7-49e6-4e23-8fab-bf4dbc4fd74c'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:16.158557 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-0ac633f7-49e6-4e23-8fab-bf4dbc4fd74c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4d75c1b48e94d58bbacac8c9356c5a4', 'request_id': 'req-0ac633f7-49e6-4e23-8fab-bf4dbc4fd74c'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:16.173166 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-0ac633f7-49e6-4e23-8fab-bf4dbc4fd74c admin admin] The token cache is being invalidate because project a4d75c1b48e94d58bbacac8c9356c5a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:51:16.173603 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-0ac633f7-49e6-4e23-8fab-bf4dbc4fd74c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:16.174369 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-0ac633f7-49e6-4e23-8fab-bf4dbc4fd74c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:16.176467 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1134/2295] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:51:15 2022] DELETE /identity/v3/projects/a4d75c1b48e94d58bbacac8c9356c5a4 => generated 0 bytes in 663 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:51:16.193724 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac633f7-49e6-4e23-8fab-bf4dbc4fd74c admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:16.193724 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-0ac633f7-49e6-4e23-8fab-bf4dbc4fd74c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:16.233519 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac633f7-49e6-4e23-8fab-bf4dbc4fd74c admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:16.244923 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7332b9f-f845-48d6-9bc8-35c189463cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:16.249832 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7332b9f-f845-48d6-9bc8-35c189463cc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:16.249832 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7332b9f-f845-48d6-9bc8-35c189463cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:16.249832 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7332b9f-f845-48d6-9bc8-35c189463cc5 admin admin] PATH_INFO: `/v3/users/9d143d51e006479e9badfd81886ada81` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:16.249832 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7332b9f-f845-48d6-9bc8-35c189463cc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d143d51e006479e9badfd81886ada81)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:16.251496 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7332b9f-f845-48d6-9bc8-35c189463cc5 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:16.275832 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1162/2296] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:51:15 2022] POST /identity/v3/projects => generated 385 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:51:16.275832 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc004570-a42c-4a62-ac67-7c0f7f545d4e admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:16.275832 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-dc004570-a42c-4a62-ac67-7c0f7f545d4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:16.346620 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc004570-a42c-4a62-ac67-7c0f7f545d4e admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:16.353158 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac68633-9b51-4d55-a1c3-ffc9e477c602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:16.354887 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac68633-9b51-4d55-a1c3-ffc9e477c602 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:16.354887 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac68633-9b51-4d55-a1c3-ffc9e477c602 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:16.354887 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac68633-9b51-4d55-a1c3-ffc9e477c602 admin admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:16.355153 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac68633-9b51-4d55-a1c3-ffc9e477c602 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:16.357024 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac68633-9b51-4d55-a1c3-ffc9e477c602 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:16.628997 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1163/2297] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:51:16 2022] POST /identity/v3/users => generated 293 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:51:16.633446 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac68633-9b51-4d55-a1c3-ffc9e477c602 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:16.646054 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac68633-9b51-4d55-a1c3-ffc9e477c602 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:16.652198 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eab4891-040c-4c78-b39b-f7834e6657a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:16.652859 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab4891-040c-4c78-b39b-f7834e6657a6 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:16.653192 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab4891-040c-4c78-b39b-f7834e6657a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:16.653465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab4891-040c-4c78-b39b-f7834e6657a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:16.654110 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eab4891-040c-4c78-b39b-f7834e6657a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:16.655708 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eab4891-040c-4c78-b39b-f7834e6657a6 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:16.673993 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1164/2298] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:51:16.674395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-f7332b9f-f845-48d6-9bc8-35c189463cc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d143d51e006479e9badfd81886ada81'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:16.675322 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eab4891-040c-4c78-b39b-f7834e6657a6 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:16.690098 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-f7332b9f-f845-48d6-9bc8-35c189463cc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d143d51e006479e9badfd81886ada81'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:16.698547 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eab4891-040c-4c78-b39b-f7834e6657a6 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:16.701580 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d2b8e-b441-469a-b165-21ad3ce42644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:16.702307 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d2b8e-b441-469a-b165-21ad3ce42644 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:16.702718 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d2b8e-b441-469a-b165-21ad3ce42644 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:16.703017 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d2b8e-b441-469a-b165-21ad3ce42644 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:16.704064 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892d2b8e-b441-469a-b165-21ad3ce42644 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:16.705583 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892d2b8e-b441-469a-b165-21ad3ce42644 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:16.706361 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-f7332b9f-f845-48d6-9bc8-35c189463cc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d143d51e006479e9badfd81886ada81'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:16.727939 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:51:16.731541 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1165/2299] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:51:16.735892 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d2b8e-b441-469a-b165-21ad3ce42644 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:16.747900 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d2b8e-b441-469a-b165-21ad3ce42644 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:16.753485 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4003ccff-3256-467c-8ebd-a1b3d6cf8bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:16.754234 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003ccff-3256-467c-8ebd-a1b3d6cf8bf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:16.754519 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003ccff-3256-467c-8ebd-a1b3d6cf8bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:16.754892 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003ccff-3256-467c-8ebd-a1b3d6cf8bf4 admin admin] PATH_INFO: `/v3/projects/6468612f8d4e4172a8bcc16f0ba73e5a/users/8cad1ee24ee84aa18da56d1145eac98a/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:16.776058 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4003ccff-3256-467c-8ebd-a1b3d6cf8bf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6468612f8d4e4172a8bcc16f0ba73e5a, user_id=8cad1ee24ee84aa18da56d1145eac98a, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:16.779164 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4003ccff-3256-467c-8ebd-a1b3d6cf8bf4 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:16.826955 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-f7332b9f-f845-48d6-9bc8-35c189463cc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d143d51e006479e9badfd81886ada81'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:16.827796 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-f7332b9f-f845-48d6-9bc8-35c189463cc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d143d51e006479e9badfd81886ada81'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:16.839793 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1135/2300] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:51:16 2022] DELETE /identity/v3/users/9d143d51e006479e9badfd81886ada81 => generated 0 bytes in 656 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:51:16.864656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1166/2301] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:51:16 2022] PUT /identity/v3/projects/6468612f8d4e4172a8bcc16f0ba73e5a/users/8cad1ee24ee84aa18da56d1145eac98a/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:51:16.871467 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc18a9-7646-4e47-b2fa-c7b8a14979b7 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:16.871754 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc18a9-7646-4e47-b2fa-c7b8a14979b7 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:16.872033 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc18a9-7646-4e47-b2fa-c7b8a14979b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:16.894070 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-19fc18a9-7646-4e47-b2fa-c7b8a14979b7 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:51:16.894683 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-19fc18a9-7646-4e47-b2fa-c7b8a14979b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:16.963241 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1136/2302] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:51:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:51:16.973334 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4003ccff-3256-467c-8ebd-a1b3d6cf8bf4 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:16.975268 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-4003ccff-3256-467c-8ebd-a1b3d6cf8bf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:17.050199 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4003ccff-3256-467c-8ebd-a1b3d6cf8bf4 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:17.055156 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75826004-8b60-442d-936b-21237d64d24d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:17.055845 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75826004-8b60-442d-936b-21237d64d24d service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:17.056198 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75826004-8b60-442d-936b-21237d64d24d service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:17.056475 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75826004-8b60-442d-936b-21237d64d24d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:17.075104 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75826004-8b60-442d-936b-21237d64d24d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:17.076288 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75826004-8b60-442d-936b-21237d64d24d service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:17.111244 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1167/2303] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:51:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:51:17.222006 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:51:17.624046 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fc18a9-7646-4e47-b2fa-c7b8a14979b7 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:17.625593 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-19fc18a9-7646-4e47-b2fa-c7b8a14979b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:17.641429 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fc18a9-7646-4e47-b2fa-c7b8a14979b7 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:17.647883 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9912bf1-8897-4a6b-a78f-ae1ef96b837e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:17.648825 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9912bf1-8897-4a6b-a78f-ae1ef96b837e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:17.649160 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9912bf1-8897-4a6b-a78f-ae1ef96b837e admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:17.649675 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9912bf1-8897-4a6b-a78f-ae1ef96b837e admin admin] PATH_INFO: `/v3/projects/548f1efb15d2440c82779e412f62fc50` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:17.650877 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9912bf1-8897-4a6b-a78f-ae1ef96b837e admin admin] RBAC: Authorizing `identity:delete_project(project_id=548f1efb15d2440c82779e412f62fc50)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:17.653274 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9912bf1-8897-4a6b-a78f-ae1ef96b837e admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:17.798833 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-e9912bf1-8897-4a6b-a78f-ae1ef96b837e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '548f1efb15d2440c82779e412f62fc50', 'request_id': 'req-e9912bf1-8897-4a6b-a78f-ae1ef96b837e'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:17.872647 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-e9912bf1-8897-4a6b-a78f-ae1ef96b837e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '548f1efb15d2440c82779e412f62fc50', 'request_id': 'req-e9912bf1-8897-4a6b-a78f-ae1ef96b837e'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:17.884974 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-e9912bf1-8897-4a6b-a78f-ae1ef96b837e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '548f1efb15d2440c82779e412f62fc50', 'request_id': 'req-e9912bf1-8897-4a6b-a78f-ae1ef96b837e'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:17.900428 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-e9912bf1-8897-4a6b-a78f-ae1ef96b837e admin admin] The token cache is being invalidate because project 548f1efb15d2440c82779e412f62fc50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:51:17.900791 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-e9912bf1-8897-4a6b-a78f-ae1ef96b837e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:17.901463 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-e9912bf1-8897-4a6b-a78f-ae1ef96b837e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:17.904157 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1137/2304] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:51:17 2022] DELETE /identity/v3/projects/548f1efb15d2440c82779e412f62fc50 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:51:17.910457 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cedb2a3-e9a3-4b4d-8b9b-1fc23f6b1c45 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:17.910457 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cedb2a3-e9a3-4b4d-8b9b-1fc23f6b1c45 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:17.910457 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cedb2a3-e9a3-4b4d-8b9b-1fc23f6b1c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:17.954315 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-4cedb2a3-e9a3-4b4d-8b9b-1fc23f6b1c45 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:51:17.955873 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-4cedb2a3-e9a3-4b4d-8b9b-1fc23f6b1c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:18.078575 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1168/2305] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:51:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:51:18.173611 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12292611-86cd-4ecf-a6b0-3e9b6fcec71c None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:18.174129 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12292611-86cd-4ecf-a6b0-3e9b6fcec71c None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:18.174568 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12292611-86cd-4ecf-a6b0-3e9b6fcec71c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:18.218655 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-12292611-86cd-4ecf-a6b0-3e9b6fcec71c None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:51:18.219991 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-12292611-86cd-4ecf-a6b0-3e9b6fcec71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:18.255651 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1138/2306] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:51:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:51:18.257450 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cedb2a3-e9a3-4b4d-8b9b-1fc23f6b1c45 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:18.267829 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cedb2a3-e9a3-4b4d-8b9b-1fc23f6b1c45 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:18.277562 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b58b5a0-fce9-4557-8864-0c6796d13803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:18.278419 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b58b5a0-fce9-4557-8864-0c6796d13803 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:18.278859 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b58b5a0-fce9-4557-8864-0c6796d13803 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:18.279771 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b58b5a0-fce9-4557-8864-0c6796d13803 admin admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:18.280623 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b58b5a0-fce9-4557-8864-0c6796d13803 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:51:18.281122 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b58b5a0-fce9-4557-8864-0c6796d13803 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:18.282905 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b58b5a0-fce9-4557-8864-0c6796d13803 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:18.296580 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1169/2307] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:51:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:51:18.305253 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12292611-86cd-4ecf-a6b0-3e9b6fcec71c None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:18.315009 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12292611-86cd-4ecf-a6b0-3e9b6fcec71c None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:18.322073 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f28dd2-24a5-4597-91ec-8867984046b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:18.330968 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f28dd2-24a5-4597-91ec-8867984046b1 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:18.334067 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f28dd2-24a5-4597-91ec-8867984046b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:18.334067 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f28dd2-24a5-4597-91ec-8867984046b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:18.334067 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f28dd2-24a5-4597-91ec-8867984046b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:18.335599 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f28dd2-24a5-4597-91ec-8867984046b1 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:18.460506 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1139/2308] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:51:18 2022] POST /identity/v3/projects => generated 391 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:51:18.469089 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b58b5a0-fce9-4557-8864-0c6796d13803 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:18.488175 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b58b5a0-fce9-4557-8864-0c6796d13803 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:18.551234 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:51:18.579320 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea7f4b3-a6a5-48f9-8c66-3031a4d783a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:18.585335 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7f4b3-a6a5-48f9-8c66-3031a4d783a3 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:18.585626 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7f4b3-a6a5-48f9-8c66-3031a4d783a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:18.585913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7f4b3-a6a5-48f9-8c66-3031a4d783a3 admin admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:18.586614 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea7f4b3-a6a5-48f9-8c66-3031a4d783a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:18.592101 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea7f4b3-a6a5-48f9-8c66-3031a4d783a3 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:18.739284 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1170/2309] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:51:18 2022] POST /identity/v3/users => generated 296 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:51:18.744373 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f28dd2-24a5-4597-91ec-8867984046b1 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:18.761539 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f28dd2-24a5-4597-91ec-8867984046b1 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:18.799028 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873417cb-6140-4a9e-a671-a5a330613199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:18.799942 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873417cb-6140-4a9e-a671-a5a330613199 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:18.800688 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873417cb-6140-4a9e-a671-a5a330613199 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:18.811642 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873417cb-6140-4a9e-a671-a5a330613199 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:18.811642 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873417cb-6140-4a9e-a671-a5a330613199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:18.811642 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873417cb-6140-4a9e-a671-a5a330613199 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:18.858159 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1140/2310] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:51:18 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:51:18.861890 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea7f4b3-a6a5-48f9-8c66-3031a4d783a3 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:18.878689 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea7f4b3-a6a5-48f9-8c66-3031a4d783a3 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:18.889775 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b721d97-7193-4dde-ae70-a13bbd2f45d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:18.890912 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b721d97-7193-4dde-ae70-a13bbd2f45d1 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:18.891266 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b721d97-7193-4dde-ae70-a13bbd2f45d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:18.891924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b721d97-7193-4dde-ae70-a13bbd2f45d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:18.893080 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b721d97-7193-4dde-ae70-a13bbd2f45d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:18.895819 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b721d97-7193-4dde-ae70-a13bbd2f45d1 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:18.916532 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1171/2311] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:51:18 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:51:18.921083 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873417cb-6140-4a9e-a671-a5a330613199 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:18.932473 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873417cb-6140-4a9e-a671-a5a330613199 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:18.937731 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40cfc59-2688-483b-a30d-3a1f353b7b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:18.945208 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40cfc59-2688-483b-a30d-3a1f353b7b66 admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:18.946165 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40cfc59-2688-483b-a30d-3a1f353b7b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:18.946567 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40cfc59-2688-483b-a30d-3a1f353b7b66 admin admin] PATH_INFO: `/v3/projects/a1db603f0b4f4654bf3b5c25d202d3dd/users/9d71c39b32924de68e2aced068c7401d/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:18.967607 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40cfc59-2688-483b-a30d-3a1f353b7b66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1db603f0b4f4654bf3b5c25d202d3dd, user_id=9d71c39b32924de68e2aced068c7401d, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:18.970423 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40cfc59-2688-483b-a30d-3a1f353b7b66 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:19.053756 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:51:19.065438 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1141/2312] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:51:18 2022] PUT /identity/v3/projects/a1db603f0b4f4654bf3b5c25d202d3dd/users/9d71c39b32924de68e2aced068c7401d/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:51:19.073199 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e929ff4-e302-48b9-90db-c5525f30443c None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:19.073525 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e929ff4-e302-48b9-90db-c5525f30443c None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:19.073848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e929ff4-e302-48b9-90db-c5525f30443c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:19.108975 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-3e929ff4-e302-48b9-90db-c5525f30443c None None] MFA Rules not processed for user `9d71c39b32924de68e2aced068c7401d`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:51:19.110258 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-3e929ff4-e302-48b9-90db-c5525f30443c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:19.213126 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1172/2313] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:51:19 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:51:19.232873 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40cfc59-2688-483b-a30d-3a1f353b7b66 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:19.235242 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-f40cfc59-2688-483b-a30d-3a1f353b7b66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:19.355240 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40cfc59-2688-483b-a30d-3a1f353b7b66 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:19.378317 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23e908d-3e73-47d7-bcc1-6fea6823803e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:19.379878 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e908d-3e73-47d7-bcc1-6fea6823803e service cinder] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:19.379878 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e908d-3e73-47d7-bcc1-6fea6823803e service cinder] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:19.379878 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e908d-3e73-47d7-bcc1-6fea6823803e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:19.404207 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23e908d-3e73-47d7-bcc1-6fea6823803e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:19.405786 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23e908d-3e73-47d7-bcc1-6fea6823803e service cinder] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:19.442676 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1142/2314] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:51:19 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:51:19.659198 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:51:24.681872 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa648b-8619-4f09-b453-cb46562232db None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:24.682776 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa648b-8619-4f09-b453-cb46562232db None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:24.683059 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa648b-8619-4f09-b453-cb46562232db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:24.719541 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-b6fa648b-8619-4f09-b453-cb46562232db None None] MFA Rules not processed for user `8cad1ee24ee84aa18da56d1145eac98a`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:51:24.734137 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-b6fa648b-8619-4f09-b453-cb46562232db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:24.838854 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1173/2315] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:51:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:51:24.843456 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23e908d-3e73-47d7-bcc1-6fea6823803e service cinder] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:24.844968 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-b23e908d-3e73-47d7-bcc1-6fea6823803e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:24.917146 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23e908d-3e73-47d7-bcc1-6fea6823803e service cinder] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:24.921795 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae2048c-1a43-4220-b3bb-674951d1267d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:24.926682 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2048c-1a43-4220-b3bb-674951d1267d admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:24.926682 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2048c-1a43-4220-b3bb-674951d1267d admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:24.926682 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2048c-1a43-4220-b3bb-674951d1267d admin admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:24.926682 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae2048c-1a43-4220-b3bb-674951d1267d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:24.926682 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae2048c-1a43-4220-b3bb-674951d1267d admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:25.034286 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1143/2316] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:51:24 2022] POST /identity/v3/projects => generated 385 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:51:25.039358 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fa648b-8619-4f09-b453-cb46562232db None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:25.050482 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fa648b-8619-4f09-b453-cb46562232db None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:25.120904 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a03b63b-2b21-41ee-942b-730edd5df1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:25.121530 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a03b63b-2b21-41ee-942b-730edd5df1cf admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:25.121809 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a03b63b-2b21-41ee-942b-730edd5df1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:25.122079 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a03b63b-2b21-41ee-942b-730edd5df1cf admin admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:25.122730 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a03b63b-2b21-41ee-942b-730edd5df1cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:25.133182 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a03b63b-2b21-41ee-942b-730edd5df1cf admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:25.269661 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1174/2317] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:51:25 2022] POST /identity/v3/users => generated 299 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:51:25.275135 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae2048c-1a43-4220-b3bb-674951d1267d admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:25.287179 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae2048c-1a43-4220-b3bb-674951d1267d admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:25.292771 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e5af6e-f13e-45fa-bdef-4f8688ff7175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:25.293384 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5af6e-f13e-45fa-bdef-4f8688ff7175 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:25.293655 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5af6e-f13e-45fa-bdef-4f8688ff7175 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:25.293973 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5af6e-f13e-45fa-bdef-4f8688ff7175 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:25.294625 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e5af6e-f13e-45fa-bdef-4f8688ff7175 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:25.296222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e5af6e-f13e-45fa-bdef-4f8688ff7175 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:25.308172 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1144/2318] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:51:25.312308 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a03b63b-2b21-41ee-942b-730edd5df1cf admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:25.324381 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a03b63b-2b21-41ee-942b-730edd5df1cf admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:25.329960 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c288fbc-28ad-47ed-8b01-7abbb5e94c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:25.330816 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c288fbc-28ad-47ed-8b01-7abbb5e94c90 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:25.331103 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c288fbc-28ad-47ed-8b01-7abbb5e94c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:25.331380 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c288fbc-28ad-47ed-8b01-7abbb5e94c90 admin admin] PATH_INFO: `/v3/projects/550e5821300144f68160dd9a48f9ef2d/users/99078ca3f1604e169781f89ac8ca5182/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:25.353104 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c288fbc-28ad-47ed-8b01-7abbb5e94c90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=550e5821300144f68160dd9a48f9ef2d, user_id=99078ca3f1604e169781f89ac8ca5182, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:25.356592 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c288fbc-28ad-47ed-8b01-7abbb5e94c90 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:25.428906 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1175/2319] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:51:25 2022] PUT /identity/v3/projects/550e5821300144f68160dd9a48f9ef2d/users/99078ca3f1604e169781f89ac8ca5182/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:51:25.432374 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e5af6e-f13e-45fa-bdef-4f8688ff7175 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:25.443174 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e5af6e-f13e-45fa-bdef-4f8688ff7175 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:25.524859 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5fee45-25d1-499e-a67c-d7757950a274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:25.525481 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5fee45-25d1-499e-a67c-d7757950a274 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:25.525758 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5fee45-25d1-499e-a67c-d7757950a274 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:25.526028 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5fee45-25d1-499e-a67c-d7757950a274 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:25.526677 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5fee45-25d1-499e-a67c-d7757950a274 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:25.528282 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5fee45-25d1-499e-a67c-d7757950a274 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:25.557128 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1145/2320] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:51:25.557641 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c288fbc-28ad-47ed-8b01-7abbb5e94c90 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:25.572359 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c288fbc-28ad-47ed-8b01-7abbb5e94c90 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:25.579723 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482f3d27-3f4e-49d8-a512-6c0623fe5976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:25.580406 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482f3d27-3f4e-49d8-a512-6c0623fe5976 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:25.580687 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482f3d27-3f4e-49d8-a512-6c0623fe5976 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:25.581020 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482f3d27-3f4e-49d8-a512-6c0623fe5976 admin admin] PATH_INFO: `/v3/projects/550e5821300144f68160dd9a48f9ef2d/users/99078ca3f1604e169781f89ac8ca5182/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:25.583474 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482f3d27-3f4e-49d8-a512-6c0623fe5976 admin admin] RBAC: Authorizing `identity:create_grant(project_id=550e5821300144f68160dd9a48f9ef2d, user_id=99078ca3f1604e169781f89ac8ca5182, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:25.587704 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482f3d27-3f4e-49d8-a512-6c0623fe5976 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:25.602201 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1176/2321] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:51:25 2022] PUT /identity/v3/projects/550e5821300144f68160dd9a48f9ef2d/users/99078ca3f1604e169781f89ac8ca5182/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:51:26.062098 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c487596-c911-4439-a72d-033f13ba841e None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:26.062656 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c487596-c911-4439-a72d-033f13ba841e None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:26.062750 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c487596-c911-4439-a72d-033f13ba841e None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:26.064859 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1146/2322] 10.222.0.52 () {54 vars in 961 bytes} [Mon Aug 29 22:51:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:51:26.073126 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e0a94-fb6d-4358-9eef-f4085670ebc1 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:26.079884 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e0a94-fb6d-4358-9eef-f4085670ebc1 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:26.079884 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e0a94-fb6d-4358-9eef-f4085670ebc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:26.094433 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-668e0a94-fb6d-4358-9eef-f4085670ebc1 None None] MFA Rules not processed for user `d8fd0f6e5b144d2ca78b8d96722502a7`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:51:26.095739 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-668e0a94-fb6d-4358-9eef-f4085670ebc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:26.111218 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:51:26.194078 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1177/2323] 10.222.0.52 () {58 vars in 1058 bytes} [Mon Aug 29 22:51:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:51:26.201752 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c487596-c911-4439-a72d-033f13ba841e None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:26.220656 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c487596-c911-4439-a72d-033f13ba841e None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:26.229407 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20170efa-ea16-4e6e-bb16-8a819ee1322a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8fd0f6e5b144d2ca78b8d96722502a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:26.230402 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20170efa-ea16-4e6e-bb16-8a819ee1322a service placement] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:26.231109 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20170efa-ea16-4e6e-bb16-8a819ee1322a service placement] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:26.231569 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20170efa-ea16-4e6e-bb16-8a819ee1322a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:26.243324 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-20170efa-ea16-4e6e-bb16-8a819ee1322a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:26.275923 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20170efa-ea16-4e6e-bb16-8a819ee1322a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:26.281720 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20170efa-ea16-4e6e-bb16-8a819ee1322a service placement] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:26.289198 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:51:26.298169 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1147/2324] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:51:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:51:28.552112 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e0a94-fb6d-4358-9eef-f4085670ebc1 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:28.553916 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-668e0a94-fb6d-4358-9eef-f4085670ebc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:28.685113 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e0a94-fb6d-4358-9eef-f4085670ebc1 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:28.691003 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cfa409-77ef-4577-86fb-e1f8c676b6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:28.691951 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cfa409-77ef-4577-86fb-e1f8c676b6e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:28.692431 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cfa409-77ef-4577-86fb-e1f8c676b6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:28.692913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cfa409-77ef-4577-86fb-e1f8c676b6e9 admin admin] PATH_INFO: `/v3/users/80771602ae7f4e27b9a4d4e9d524e37f` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:28.695039 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cfa409-77ef-4577-86fb-e1f8c676b6e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80771602ae7f4e27b9a4d4e9d524e37f)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:28.697252 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cfa409-77ef-4577-86fb-e1f8c676b6e9 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:29.039381 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-34cfa409-77ef-4577-86fb-e1f8c676b6e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80771602ae7f4e27b9a4d4e9d524e37f'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:29.116219 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-34cfa409-77ef-4577-86fb-e1f8c676b6e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80771602ae7f4e27b9a4d4e9d524e37f'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:29.133539 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-34cfa409-77ef-4577-86fb-e1f8c676b6e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80771602ae7f4e27b9a4d4e9d524e37f'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:29.151290 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-34cfa409-77ef-4577-86fb-e1f8c676b6e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80771602ae7f4e27b9a4d4e9d524e37f'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:29.152406 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-34cfa409-77ef-4577-86fb-e1f8c676b6e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80771602ae7f4e27b9a4d4e9d524e37f'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:29.161862 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1178/2325] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:51:28 2022] DELETE /identity/v3/users/80771602ae7f4e27b9a4d4e9d524e37f => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:51:29.482932 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20170efa-ea16-4e6e-bb16-8a819ee1322a service placement] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:29.494916 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20170efa-ea16-4e6e-bb16-8a819ee1322a service placement] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:29.575866 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91699794-2bf6-4e9d-960e-54c739e18381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:29.576541 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91699794-2bf6-4e9d-960e-54c739e18381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:29.576855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91699794-2bf6-4e9d-960e-54c739e18381 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:29.577125 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91699794-2bf6-4e9d-960e-54c739e18381 admin admin] PATH_INFO: `/v3/projects/50a332bdc684445cb6d1c937dcee5487` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:29.579157 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91699794-2bf6-4e9d-960e-54c739e18381 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50a332bdc684445cb6d1c937dcee5487)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:29.581734 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91699794-2bf6-4e9d-960e-54c739e18381 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:29.730971 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-91699794-2bf6-4e9d-960e-54c739e18381 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50a332bdc684445cb6d1c937dcee5487', 'request_id': 'req-91699794-2bf6-4e9d-960e-54c739e18381'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:29.858022 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-91699794-2bf6-4e9d-960e-54c739e18381 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50a332bdc684445cb6d1c937dcee5487', 'request_id': 'req-91699794-2bf6-4e9d-960e-54c739e18381'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:29.866168 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-91699794-2bf6-4e9d-960e-54c739e18381 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50a332bdc684445cb6d1c937dcee5487', 'request_id': 'req-91699794-2bf6-4e9d-960e-54c739e18381'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:29.882348 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-91699794-2bf6-4e9d-960e-54c739e18381 admin admin] The token cache is being invalidate because project 50a332bdc684445cb6d1c937dcee5487 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:51:29.882585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-91699794-2bf6-4e9d-960e-54c739e18381 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:29.883228 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-91699794-2bf6-4e9d-960e-54c739e18381 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:29.884992 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1148/2326] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:51:29 2022] DELETE /identity/v3/projects/50a332bdc684445cb6d1c937dcee5487 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:51:29.888775 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cfa409-77ef-4577-86fb-e1f8c676b6e9 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:29.891213 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-34cfa409-77ef-4577-86fb-e1f8c676b6e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:29.904375 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cfa409-77ef-4577-86fb-e1f8c676b6e9 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:29.947659 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:51:29.981197 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9a664e-ef7e-4d56-976f-db5b3bfc9ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:29.981766 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a664e-ef7e-4d56-976f-db5b3bfc9ee9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:29.982098 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a664e-ef7e-4d56-976f-db5b3bfc9ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:29.982383 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a664e-ef7e-4d56-976f-db5b3bfc9ee9 admin admin] PATH_INFO: `/v3/users/3b581d01552a40dda1838870c8b8e2ce` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:29.983734 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9a664e-ef7e-4d56-976f-db5b3bfc9ee9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b581d01552a40dda1838870c8b8e2ce)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:29.985694 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9a664e-ef7e-4d56-976f-db5b3bfc9ee9 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:30.255272 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-df9a664e-ef7e-4d56-976f-db5b3bfc9ee9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b581d01552a40dda1838870c8b8e2ce'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:30.352905 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-df9a664e-ef7e-4d56-976f-db5b3bfc9ee9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b581d01552a40dda1838870c8b8e2ce'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:30.374676 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-df9a664e-ef7e-4d56-976f-db5b3bfc9ee9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b581d01552a40dda1838870c8b8e2ce'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:30.383221 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-df9a664e-ef7e-4d56-976f-db5b3bfc9ee9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b581d01552a40dda1838870c8b8e2ce'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:30.384806 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-df9a664e-ef7e-4d56-976f-db5b3bfc9ee9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b581d01552a40dda1838870c8b8e2ce'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:30.392353 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1179/2327] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:51:29 2022] DELETE /identity/v3/users/3b581d01552a40dda1838870c8b8e2ce => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:51:30.816556 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91699794-2bf6-4e9d-960e-54c739e18381 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:30.819664 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-91699794-2bf6-4e9d-960e-54c739e18381 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:30.869368 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91699794-2bf6-4e9d-960e-54c739e18381 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:30.985579 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfc1ada-f835-48b6-8ff4-caa20895c74c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:30.986330 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc1ada-f835-48b6-8ff4-caa20895c74c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:30.986621 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc1ada-f835-48b6-8ff4-caa20895c74c admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:30.986924 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc1ada-f835-48b6-8ff4-caa20895c74c admin admin] PATH_INFO: `/v3/projects/49c61540797b4ecaa717cfa6d884c01f` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:30.989425 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfc1ada-f835-48b6-8ff4-caa20895c74c admin admin] RBAC: Authorizing `identity:delete_project(project_id=49c61540797b4ecaa717cfa6d884c01f)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:30.991947 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfc1ada-f835-48b6-8ff4-caa20895c74c admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:31.174590 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:51:31.194202 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-1cfc1ada-f835-48b6-8ff4-caa20895c74c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49c61540797b4ecaa717cfa6d884c01f', 'request_id': 'req-1cfc1ada-f835-48b6-8ff4-caa20895c74c'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:31.392532 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-1cfc1ada-f835-48b6-8ff4-caa20895c74c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49c61540797b4ecaa717cfa6d884c01f', 'request_id': 'req-1cfc1ada-f835-48b6-8ff4-caa20895c74c'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:31.402888 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-1cfc1ada-f835-48b6-8ff4-caa20895c74c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49c61540797b4ecaa717cfa6d884c01f', 'request_id': 'req-1cfc1ada-f835-48b6-8ff4-caa20895c74c'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:31.440786 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-1cfc1ada-f835-48b6-8ff4-caa20895c74c admin admin] The token cache is being invalidate because project 49c61540797b4ecaa717cfa6d884c01f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:51:31.441819 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-1cfc1ada-f835-48b6-8ff4-caa20895c74c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:31.442916 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-1cfc1ada-f835-48b6-8ff4-caa20895c74c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:31.446136 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1149/2328] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:51:30 2022] DELETE /identity/v3/projects/49c61540797b4ecaa717cfa6d884c01f => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:51:31.450929 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e0cfc-8534-477b-bb6a-4f9643793270 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:31.451400 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e0cfc-8534-477b-bb6a-4f9643793270 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:31.451512 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e0cfc-8534-477b-bb6a-4f9643793270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:31.480773 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-792e0cfc-8534-477b-bb6a-4f9643793270 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:51:31.482511 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-792e0cfc-8534-477b-bb6a-4f9643793270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:31.572484 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1180/2329] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:51:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:51:31.580819 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e56fa-ecc9-4e33-8357-e9b63c33686f None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:31.581451 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e56fa-ecc9-4e33-8357-e9b63c33686f None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:31.583083 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e56fa-ecc9-4e33-8357-e9b63c33686f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:31.613819 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-476e56fa-ecc9-4e33-8357-e9b63c33686f None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:51:31.616334 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-476e56fa-ecc9-4e33-8357-e9b63c33686f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:31.625274 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1150/2330] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:51:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:51:31.630331 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e0cfc-8534-477b-bb6a-4f9643793270 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:31.642613 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e0cfc-8534-477b-bb6a-4f9643793270 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:31.649878 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4aca96-f954-4e09-9add-90fec57439f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:31.650877 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4aca96-f954-4e09-9add-90fec57439f9 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:31.651342 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4aca96-f954-4e09-9add-90fec57439f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:31.651797 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4aca96-f954-4e09-9add-90fec57439f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:31.653006 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4aca96-f954-4e09-9add-90fec57439f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:51:31.653494 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4aca96-f954-4e09-9add-90fec57439f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:31.655245 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4aca96-f954-4e09-9add-90fec57439f9 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:31.671828 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1181/2331] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:51:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:51:31.676165 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476e56fa-ecc9-4e33-8357-e9b63c33686f None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:31.689800 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476e56fa-ecc9-4e33-8357-e9b63c33686f None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:31.695870 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942b089f-7535-48e5-b893-986a71cc03a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:31.696485 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b089f-7535-48e5-b893-986a71cc03a3 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:31.697515 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b089f-7535-48e5-b893-986a71cc03a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:31.697515 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b089f-7535-48e5-b893-986a71cc03a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:31.697885 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942b089f-7535-48e5-b893-986a71cc03a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:31.699762 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942b089f-7535-48e5-b893-986a71cc03a3 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:31.872267 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1151/2332] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:51:31 2022] POST /identity/v3/projects => generated 377 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:51:31.873837 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4aca96-f954-4e09-9add-90fec57439f9 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:31.887502 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4aca96-f954-4e09-9add-90fec57439f9 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:31.937578 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:51:31.956553 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9acc4e-13bb-4dec-9bb1-6ed5ee6be043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:31.958305 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9acc4e-13bb-4dec-9bb1-6ed5ee6be043 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:31.959760 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9acc4e-13bb-4dec-9bb1-6ed5ee6be043 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:31.959760 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9acc4e-13bb-4dec-9bb1-6ed5ee6be043 admin admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:31.965518 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9acc4e-13bb-4dec-9bb1-6ed5ee6be043 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:31.965518 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9acc4e-13bb-4dec-9bb1-6ed5ee6be043 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:32.087181 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1182/2333] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:51:31 2022] POST /identity/v3/users => generated 289 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:51:32.090984 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942b089f-7535-48e5-b893-986a71cc03a3 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:32.109380 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942b089f-7535-48e5-b893-986a71cc03a3 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:32.112320 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8a0c33-4797-46e5-99f0-70d479897714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:32.113133 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a0c33-4797-46e5-99f0-70d479897714 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:32.113479 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a0c33-4797-46e5-99f0-70d479897714 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:32.113825 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a0c33-4797-46e5-99f0-70d479897714 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:32.114558 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8a0c33-4797-46e5-99f0-70d479897714 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:32.117229 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8a0c33-4797-46e5-99f0-70d479897714 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:32.132522 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1152/2334] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:51:32 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:51:32.138132 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9acc4e-13bb-4dec-9bb1-6ed5ee6be043 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:32.150639 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9acc4e-13bb-4dec-9bb1-6ed5ee6be043 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:32.157925 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09312e2f-a368-4aa3-bd15-3470fdb1ff88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:32.158528 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09312e2f-a368-4aa3-bd15-3470fdb1ff88 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:32.160576 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09312e2f-a368-4aa3-bd15-3470fdb1ff88 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:32.160576 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09312e2f-a368-4aa3-bd15-3470fdb1ff88 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:32.160576 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09312e2f-a368-4aa3-bd15-3470fdb1ff88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:32.161393 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09312e2f-a368-4aa3-bd15-3470fdb1ff88 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:32.174251 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1183/2335] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:51:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:51:32.180196 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8a0c33-4797-46e5-99f0-70d479897714 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:32.194967 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8a0c33-4797-46e5-99f0-70d479897714 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:32.200224 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f5794-e154-4d6f-a103-a8f1b8128b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:32.201506 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f5794-e154-4d6f-a103-a8f1b8128b0c admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:32.201740 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f5794-e154-4d6f-a103-a8f1b8128b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:32.202088 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f5794-e154-4d6f-a103-a8f1b8128b0c admin admin] PATH_INFO: `/v3/projects/a0a9cfe3881945be8b9c12fa93c94fb9/users/6f803e64a0784410a2644431b378b664/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:32.221572 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4f5794-e154-4d6f-a103-a8f1b8128b0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0a9cfe3881945be8b9c12fa93c94fb9, user_id=6f803e64a0784410a2644431b378b664, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:32.225252 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4f5794-e154-4d6f-a103-a8f1b8128b0c admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:32.366593 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1153/2336] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:51:32 2022] PUT /identity/v3/projects/a0a9cfe3881945be8b9c12fa93c94fb9/users/6f803e64a0784410a2644431b378b664/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:51:32.377799 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc88bf2-78e9-485d-a3eb-8c1d680a5954 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:32.382642 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc88bf2-78e9-485d-a3eb-8c1d680a5954 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:32.383100 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc88bf2-78e9-485d-a3eb-8c1d680a5954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:32.433448 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-bdc88bf2-78e9-485d-a3eb-8c1d680a5954 None None] MFA Rules not processed for user `6f803e64a0784410a2644431b378b664`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:51:32.434812 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-bdc88bf2-78e9-485d-a3eb-8c1d680a5954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:32.510289 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1184/2337] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:51:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:51:32.516240 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f5794-e154-4d6f-a103-a8f1b8128b0c admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:32.609700 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f5794-e154-4d6f-a103-a8f1b8128b0c admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:32.615042 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e0dfa9-50a6-4feb-bd2f-3c45246d46a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:32.618786 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e0dfa9-50a6-4feb-bd2f-3c45246d46a0 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:32.618786 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e0dfa9-50a6-4feb-bd2f-3c45246d46a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:32.619040 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e0dfa9-50a6-4feb-bd2f-3c45246d46a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:32.621568 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e0dfa9-50a6-4feb-bd2f-3c45246d46a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:32.622292 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e0dfa9-50a6-4feb-bd2f-3c45246d46a0 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:32.793827 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1154/2338] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:51:32 2022] POST /identity/v3/projects => generated 373 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:51:32.802362 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc88bf2-78e9-485d-a3eb-8c1d680a5954 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:32.814272 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc88bf2-78e9-485d-a3eb-8c1d680a5954 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:32.876799 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2524090-fbf4-42c8-ac91-9a454f40201f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:32.877678 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2524090-fbf4-42c8-ac91-9a454f40201f admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:32.878125 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2524090-fbf4-42c8-ac91-9a454f40201f admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:32.878405 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2524090-fbf4-42c8-ac91-9a454f40201f admin admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:32.879284 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2524090-fbf4-42c8-ac91-9a454f40201f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:32.881873 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2524090-fbf4-42c8-ac91-9a454f40201f admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:32.959688 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1185/2339] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:51:32 2022] POST /identity/v3/users => generated 287 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:51:32.961631 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e0dfa9-50a6-4feb-bd2f-3c45246d46a0 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:32.973936 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e0dfa9-50a6-4feb-bd2f-3c45246d46a0 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:32.985643 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a012387-cf77-4627-b072-2f6501c832ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:32.987249 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a012387-cf77-4627-b072-2f6501c832ad admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:32.990277 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a012387-cf77-4627-b072-2f6501c832ad admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:32.990277 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a012387-cf77-4627-b072-2f6501c832ad admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:32.995995 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a012387-cf77-4627-b072-2f6501c832ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:33.001009 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a012387-cf77-4627-b072-2f6501c832ad admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:33.030392 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1155/2340] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:51:32 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:51:33.034781 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2524090-fbf4-42c8-ac91-9a454f40201f admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:33.048048 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2524090-fbf4-42c8-ac91-9a454f40201f admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:33.060180 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0081c850-53ad-4719-92a0-3c973fd91c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:33.066139 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081c850-53ad-4719-92a0-3c973fd91c07 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:33.066444 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081c850-53ad-4719-92a0-3c973fd91c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:33.066732 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081c850-53ad-4719-92a0-3c973fd91c07 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:33.067444 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0081c850-53ad-4719-92a0-3c973fd91c07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:33.076754 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0081c850-53ad-4719-92a0-3c973fd91c07 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:33.093215 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1186/2341] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:51:33 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:51:33.094935 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a012387-cf77-4627-b072-2f6501c832ad admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:33.108278 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a012387-cf77-4627-b072-2f6501c832ad admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:33.119350 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de03602-aecb-42ce-b196-11fdb0a5d5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:33.120439 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de03602-aecb-42ce-b196-11fdb0a5d5cf admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:33.120950 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de03602-aecb-42ce-b196-11fdb0a5d5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:33.121445 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de03602-aecb-42ce-b196-11fdb0a5d5cf admin admin] PATH_INFO: `/v3/projects/ba87aa60bda44f9a8e4804bf6e010be4/users/562282409632477f9513bf4dc3ad40e9/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:33.144035 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de03602-aecb-42ce-b196-11fdb0a5d5cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba87aa60bda44f9a8e4804bf6e010be4, user_id=562282409632477f9513bf4dc3ad40e9, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:33.148433 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de03602-aecb-42ce-b196-11fdb0a5d5cf admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:33.275120 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a03e5a-850d-46d1-8309-7b28bbfdc4a9 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:33.275415 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a03e5a-850d-46d1-8309-7b28bbfdc4a9 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:33.275702 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a03e5a-850d-46d1-8309-7b28bbfdc4a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:33.284189 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1156/2342] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:51:33 2022] PUT /identity/v3/projects/ba87aa60bda44f9a8e4804bf6e010be4/users/562282409632477f9513bf4dc3ad40e9/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:51:33.313431 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-61a03e5a-850d-46d1-8309-7b28bbfdc4a9 None None] MFA Rules not processed for user `562282409632477f9513bf4dc3ad40e9`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:51:33.314673 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-61a03e5a-850d-46d1-8309-7b28bbfdc4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:33.333611 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:51:33.347844 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62255787-d15b-41c8-8e5a-831bd0963d68 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:33.348360 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62255787-d15b-41c8-8e5a-831bd0963d68 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:33.348877 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62255787-d15b-41c8-8e5a-831bd0963d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:33.358672 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:51:33.379980 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-62255787-d15b-41c8-8e5a-831bd0963d68 None None] MFA Rules not processed for user `99078ca3f1604e169781f89ac8ca5182`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:51:33.381574 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-62255787-d15b-41c8-8e5a-831bd0963d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:33.419393 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1187/2343] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:51:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:51:33.419964 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a03e5a-850d-46d1-8309-7b28bbfdc4a9 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:33.513985 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1157/2344] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:51:33 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:51:33.524086 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a03e5a-850d-46d1-8309-7b28bbfdc4a9 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:33.527749 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62255787-d15b-41c8-8e5a-831bd0963d68 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:33.530583 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-62255787-d15b-41c8-8e5a-831bd0963d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:33.542384 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5ba0cf-1292-4427-90e6-5c8e882d81f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:33.543020 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ba0cf-1292-4427-90e6-5c8e882d81f4 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:33.543422 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ba0cf-1292-4427-90e6-5c8e882d81f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:33.543629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ba0cf-1292-4427-90e6-5c8e882d81f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:33.544316 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5ba0cf-1292-4427-90e6-5c8e882d81f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:33.546302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5ba0cf-1292-4427-90e6-5c8e882d81f4 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:33.628079 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62255787-d15b-41c8-8e5a-831bd0963d68 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:33.641472 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1188/2345] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:51:33 2022] POST /identity/v3/projects => generated 375 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:51:33.642520 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5ba0cf-1292-4427-90e6-5c8e882d81f4 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:33.643751 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddb8936-2996-414b-a698-8afe3f2585e5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c2569e7cd774199aa5750fd6a850060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:33.644585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb8936-2996-414b-a698-8afe3f2585e5 service glance] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:33.649248 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb8936-2996-414b-a698-8afe3f2585e5 service glance] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:33.649679 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb8936-2996-414b-a698-8afe3f2585e5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:33.652000 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-7ddb8936-2996-414b-a698-8afe3f2585e5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:33.718020 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddb8936-2996-414b-a698-8afe3f2585e5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:33.719736 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddb8936-2996-414b-a698-8afe3f2585e5 service glance] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:33.730206 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1158/2346] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:51:33 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:51:33.731270 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5ba0cf-1292-4427-90e6-5c8e882d81f4 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:33.736166 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6070032a-0e7b-46c8-a203-e6af072fac07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:33.736810 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070032a-0e7b-46c8-a203-e6af072fac07 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:33.737087 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070032a-0e7b-46c8-a203-e6af072fac07 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:33.737367 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070032a-0e7b-46c8-a203-e6af072fac07 admin admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:33.738007 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6070032a-0e7b-46c8-a203-e6af072fac07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:33.739933 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6070032a-0e7b-46c8-a203-e6af072fac07 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:33.833075 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1189/2347] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:51:33 2022] POST /identity/v3/users => generated 294 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:51:33.837752 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddb8936-2996-414b-a698-8afe3f2585e5 service glance] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:33.849290 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddb8936-2996-414b-a698-8afe3f2585e5 service glance] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:33.855150 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b1b649-876f-498f-a474-a0af8fc1bb43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:33.855761 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1b649-876f-498f-a474-a0af8fc1bb43 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:33.856044 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1b649-876f-498f-a474-a0af8fc1bb43 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:33.856323 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1b649-876f-498f-a474-a0af8fc1bb43 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:33.857087 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b1b649-876f-498f-a474-a0af8fc1bb43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:33.858735 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b1b649-876f-498f-a474-a0af8fc1bb43 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:33.877892 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1159/2348] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:51:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:51:33.878600 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6070032a-0e7b-46c8-a203-e6af072fac07 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:33.898885 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6070032a-0e7b-46c8-a203-e6af072fac07 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:33.905412 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421ebb8d-dc5f-49dd-b99f-aa4bddf124e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:33.906096 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ebb8d-dc5f-49dd-b99f-aa4bddf124e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:33.908974 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ebb8d-dc5f-49dd-b99f-aa4bddf124e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:33.908974 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ebb8d-dc5f-49dd-b99f-aa4bddf124e7 admin admin] PATH_INFO: `/v3/projects/2227f20843d44004a35e39f7aa9d1f01/users/be48ce7718ff48c9b50c316e3e83a48f/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:33.924584 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421ebb8d-dc5f-49dd-b99f-aa4bddf124e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2227f20843d44004a35e39f7aa9d1f01, user_id=be48ce7718ff48c9b50c316e3e83a48f, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:33.928384 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421ebb8d-dc5f-49dd-b99f-aa4bddf124e7 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:34.024769 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1190/2349] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:51:33 2022] PUT /identity/v3/projects/2227f20843d44004a35e39f7aa9d1f01/users/be48ce7718ff48c9b50c316e3e83a48f/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:51:34.028948 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b1b649-876f-498f-a474-a0af8fc1bb43 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:34.041296 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b1b649-876f-498f-a474-a0af8fc1bb43 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:34.043641 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7fe9f-af30-4306-94e6-3b30dcd8634d None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:34.043932 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7fe9f-af30-4306-94e6-3b30dcd8634d None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:34.044209 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7fe9f-af30-4306-94e6-3b30dcd8634d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:34.065908 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-71e7fe9f-af30-4306-94e6-3b30dcd8634d None None] MFA Rules not processed for user `f77dfe78b1a8405084c06d9acee96511`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:51:34.067180 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-71e7fe9f-af30-4306-94e6-3b30dcd8634d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:34.072475 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:51:34.074453 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:51:34.108929 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f757d9b2-9a27-4edc-9fe6-405282103804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:34.109761 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757d9b2-9a27-4edc-9fe6-405282103804 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:34.110048 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757d9b2-9a27-4edc-9fe6-405282103804 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:34.110406 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757d9b2-9a27-4edc-9fe6-405282103804 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:34.110953 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f757d9b2-9a27-4edc-9fe6-405282103804 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:34.112821 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f757d9b2-9a27-4edc-9fe6-405282103804 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:34.124285 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1160/2350] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:51:34.128899 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f757d9b2-9a27-4edc-9fe6-405282103804 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:34.144487 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f757d9b2-9a27-4edc-9fe6-405282103804 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:34.150462 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb1b874-d8ba-437d-9554-7daa06d84d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:34.151443 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb1b874-d8ba-437d-9554-7daa06d84d2b admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:34.151730 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb1b874-d8ba-437d-9554-7daa06d84d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:34.155094 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb1b874-d8ba-437d-9554-7daa06d84d2b admin admin] PATH_INFO: `/v3/projects/2227f20843d44004a35e39f7aa9d1f01/users/be48ce7718ff48c9b50c316e3e83a48f/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:34.155094 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb1b874-d8ba-437d-9554-7daa06d84d2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2227f20843d44004a35e39f7aa9d1f01, user_id=be48ce7718ff48c9b50c316e3e83a48f, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:34.156878 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb1b874-d8ba-437d-9554-7daa06d84d2b admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:34.157014 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1191/2351] 10.222.0.52 () {58 vars in 1009 bytes} [Mon Aug 29 22:51:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:51:34.173552 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1161/2352] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:51:34 2022] PUT /identity/v3/projects/2227f20843d44004a35e39f7aa9d1f01/users/be48ce7718ff48c9b50c316e3e83a48f/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:51:34.173898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e7fe9f-af30-4306-94e6-3b30dcd8634d None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:34.175467 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-71e7fe9f-af30-4306-94e6-3b30dcd8634d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:34.184704 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04644c1b-9ad1-47ef-b3d2-02e73d02cea7 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:34.185091 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04644c1b-9ad1-47ef-b3d2-02e73d02cea7 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:34.185360 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04644c1b-9ad1-47ef-b3d2-02e73d02cea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:34.218691 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-04644c1b-9ad1-47ef-b3d2-02e73d02cea7 None None] MFA Rules not processed for user `be48ce7718ff48c9b50c316e3e83a48f`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:51:34.219909 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-04644c1b-9ad1-47ef-b3d2-02e73d02cea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:34.260898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e7fe9f-af30-4306-94e6-3b30dcd8634d None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:34.301816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1162/2353] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:51:34 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:51:34.314318 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04644c1b-9ad1-47ef-b3d2-02e73d02cea7 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:34.315782 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-04644c1b-9ad1-47ef-b3d2-02e73d02cea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:34.330934 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c7604-96da-46a5-9e98-68ffa65c6ccf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57c175ac90e545358746176972ff39a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:34.331554 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c7604-96da-46a5-9e98-68ffa65c6ccf service swift] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:34.331834 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c7604-96da-46a5-9e98-68ffa65c6ccf service swift] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:34.332109 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c7604-96da-46a5-9e98-68ffa65c6ccf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:34.377820 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04644c1b-9ad1-47ef-b3d2-02e73d02cea7 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:34.382439 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060732f4-83c4-4fe8-bdd6-7f9a94d7b477 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:34.383002 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060732f4-83c4-4fe8-bdd6-7f9a94d7b477 service cinder] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:34.383282 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060732f4-83c4-4fe8-bdd6-7f9a94d7b477 service cinder] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:34.383557 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060732f4-83c4-4fe8-bdd6-7f9a94d7b477 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:34.393910 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1c7604-96da-46a5-9e98-68ffa65c6ccf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:34.395503 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1c7604-96da-46a5-9e98-68ffa65c6ccf service swift] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:34.399777 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:51:34.405934 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1192/2354] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Aug 29 22:51:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 22:51:34.450931 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:51:34.455093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060732f4-83c4-4fe8-bdd6-7f9a94d7b477 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:34.456587 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060732f4-83c4-4fe8-bdd6-7f9a94d7b477 service cinder] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:34.467406 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1163/2355] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:51:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:51:38.085198 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c7604-96da-46a5-9e98-68ffa65c6ccf service swift] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:38.100026 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c7604-96da-46a5-9e98-68ffa65c6ccf service swift] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:38.110070 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3437a6c9-97a2-4164-b12c-8ab89b656873 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:38.112831 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437a6c9-97a2-4164-b12c-8ab89b656873 service cinder] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:38.112831 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437a6c9-97a2-4164-b12c-8ab89b656873 service cinder] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:38.112831 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437a6c9-97a2-4164-b12c-8ab89b656873 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:38.246688 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3437a6c9-97a2-4164-b12c-8ab89b656873 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:38.256559 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3437a6c9-97a2-4164-b12c-8ab89b656873 service cinder] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:38.277276 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1193/2356] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:51:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:51:40.326036 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cfc1ff-2187-4f42-9bf6-7e51a7090665 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:40.326960 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cfc1ff-2187-4f42-9bf6-7e51a7090665 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:40.327493 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cfc1ff-2187-4f42-9bf6-7e51a7090665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:40.356232 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-06cfc1ff-2187-4f42-9bf6-7e51a7090665 None None] MFA Rules not processed for user `f77dfe78b1a8405084c06d9acee96511`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:51:40.357732 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-06cfc1ff-2187-4f42-9bf6-7e51a7090665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:40.371615 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1164/2357] 10.222.0.52 () {58 vars in 967 bytes} [Mon Aug 29 22:51:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:51:40.394171 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3437a6c9-97a2-4164-b12c-8ab89b656873 service cinder] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:40.411258 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3437a6c9-97a2-4164-b12c-8ab89b656873 service cinder] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:40.419196 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05a0d4-3590-49c2-8b84-0bb9030f2166 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57c175ac90e545358746176972ff39a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:40.419862 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05a0d4-3590-49c2-8b84-0bb9030f2166 service swift] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:40.420146 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05a0d4-3590-49c2-8b84-0bb9030f2166 service swift] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:40.420416 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05a0d4-3590-49c2-8b84-0bb9030f2166 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:40.435137 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa05a0d4-3590-49c2-8b84-0bb9030f2166 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:40.436807 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa05a0d4-3590-49c2-8b84-0bb9030f2166 service swift] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:40.469981 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1194/2358] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Aug 29 22:51:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 22:51:40.911010 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cfc1ff-2187-4f42-9bf6-7e51a7090665 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:40.942048 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cfc1ff-2187-4f42-9bf6-7e51a7090665 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:40.987620 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd909b4-15a5-4a36-8bc4-aadb00db3c86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:40.988257 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd909b4-15a5-4a36-8bc4-aadb00db3c86 service cinder] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:40.992707 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd909b4-15a5-4a36-8bc4-aadb00db3c86 service cinder] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:40.993144 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd909b4-15a5-4a36-8bc4-aadb00db3c86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:41.045221 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd909b4-15a5-4a36-8bc4-aadb00db3c86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:41.054240 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd909b4-15a5-4a36-8bc4-aadb00db3c86 service cinder] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:41.096246 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1165/2359] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:51:40 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:51:42.867252 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05a0d4-3590-49c2-8b84-0bb9030f2166 service swift] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:42.868838 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-fa05a0d4-3590-49c2-8b84-0bb9030f2166 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:42.938413 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05a0d4-3590-49c2-8b84-0bb9030f2166 service swift] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:42.942925 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93ba794-0b3f-45ed-bdad-9ed71b81d762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:42.949108 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ba794-0b3f-45ed-bdad-9ed71b81d762 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:42.949108 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ba794-0b3f-45ed-bdad-9ed71b81d762 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:42.949108 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ba794-0b3f-45ed-bdad-9ed71b81d762 admin admin] PATH_INFO: `/v3/users/9d71c39b32924de68e2aced068c7401d` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:42.949108 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93ba794-0b3f-45ed-bdad-9ed71b81d762 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d71c39b32924de68e2aced068c7401d)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:42.949108 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93ba794-0b3f-45ed-bdad-9ed71b81d762 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:43.197969 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-f93ba794-0b3f-45ed-bdad-9ed71b81d762 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d71c39b32924de68e2aced068c7401d'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:43.275402 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-f93ba794-0b3f-45ed-bdad-9ed71b81d762 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d71c39b32924de68e2aced068c7401d'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:43.296589 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-f93ba794-0b3f-45ed-bdad-9ed71b81d762 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d71c39b32924de68e2aced068c7401d'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:43.305658 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-f93ba794-0b3f-45ed-bdad-9ed71b81d762 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d71c39b32924de68e2aced068c7401d'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:43.306654 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-f93ba794-0b3f-45ed-bdad-9ed71b81d762 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d71c39b32924de68e2aced068c7401d'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:43.317404 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1195/2360] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:51:42 2022] DELETE /identity/v3/users/9d71c39b32924de68e2aced068c7401d => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:51:43.323217 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b640a416-ecb2-4a2d-874d-e284a7a0b5cd None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:43.326600 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b640a416-ecb2-4a2d-874d-e284a7a0b5cd None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:43.326600 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b640a416-ecb2-4a2d-874d-e284a7a0b5cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:43.366189 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-b640a416-ecb2-4a2d-874d-e284a7a0b5cd None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:51:43.366189 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-b640a416-ecb2-4a2d-874d-e284a7a0b5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:43.490685 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1166/2361] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:51:43.501610 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93ba794-0b3f-45ed-bdad-9ed71b81d762 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:43.503244 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-f93ba794-0b3f-45ed-bdad-9ed71b81d762 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:43.514944 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:51:43.645168 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93ba794-0b3f-45ed-bdad-9ed71b81d762 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:43.652026 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b34e3-421b-4584-b15c-0bed1cbe6d92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:43.652826 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b34e3-421b-4584-b15c-0bed1cbe6d92 service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:43.653014 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b34e3-421b-4584-b15c-0bed1cbe6d92 service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:43.653292 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b34e3-421b-4584-b15c-0bed1cbe6d92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:43.670491 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3b34e3-421b-4584-b15c-0bed1cbe6d92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:43.672035 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3b34e3-421b-4584-b15c-0bed1cbe6d92 service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:43.690056 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1196/2362] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:51:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:51:44.237044 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b640a416-ecb2-4a2d-874d-e284a7a0b5cd None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:44.241054 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-b640a416-ecb2-4a2d-874d-e284a7a0b5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:44.280954 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b640a416-ecb2-4a2d-874d-e284a7a0b5cd None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:44.288625 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff47cc-bdfb-4e34-ad2f-02583340b61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:44.289318 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff47cc-bdfb-4e34-ad2f-02583340b61e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:44.289596 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff47cc-bdfb-4e34-ad2f-02583340b61e admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:44.289906 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff47cc-bdfb-4e34-ad2f-02583340b61e admin admin] PATH_INFO: `/v3/projects/a1db603f0b4f4654bf3b5c25d202d3dd` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:44.291131 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fff47cc-bdfb-4e34-ad2f-02583340b61e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1db603f0b4f4654bf3b5c25d202d3dd)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:44.293813 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fff47cc-bdfb-4e34-ad2f-02583340b61e admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:44.434836 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-4fff47cc-bdfb-4e34-ad2f-02583340b61e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1db603f0b4f4654bf3b5c25d202d3dd', 'request_id': 'req-4fff47cc-bdfb-4e34-ad2f-02583340b61e'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:44.509316 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-4fff47cc-bdfb-4e34-ad2f-02583340b61e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1db603f0b4f4654bf3b5c25d202d3dd', 'request_id': 'req-4fff47cc-bdfb-4e34-ad2f-02583340b61e'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:44.515780 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-4fff47cc-bdfb-4e34-ad2f-02583340b61e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1db603f0b4f4654bf3b5c25d202d3dd', 'request_id': 'req-4fff47cc-bdfb-4e34-ad2f-02583340b61e'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:44.558789 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-4fff47cc-bdfb-4e34-ad2f-02583340b61e admin admin] The token cache is being invalidate because project a1db603f0b4f4654bf3b5c25d202d3dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:51:44.559100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-4fff47cc-bdfb-4e34-ad2f-02583340b61e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:44.559748 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-4fff47cc-bdfb-4e34-ad2f-02583340b61e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:44.561555 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1167/2363] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:51:44 2022] DELETE /identity/v3/projects/a1db603f0b4f4654bf3b5c25d202d3dd => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:51:46.453430 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b34e3-421b-4584-b15c-0bed1cbe6d92 service neutron] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:46.455039 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-8b3b34e3-421b-4584-b15c-0bed1cbe6d92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:46.469100 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b34e3-421b-4584-b15c-0bed1cbe6d92 service neutron] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:46.551271 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85156f-d852-47cb-af94-261dc14e28b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:46.551913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85156f-d852-47cb-af94-261dc14e28b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:46.552191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85156f-d852-47cb-af94-261dc14e28b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:46.552481 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85156f-d852-47cb-af94-261dc14e28b0 admin admin] PATH_INFO: `/v3/users/6f803e64a0784410a2644431b378b664` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:46.553667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a85156f-d852-47cb-af94-261dc14e28b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f803e64a0784410a2644431b378b664)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:46.555562 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a85156f-d852-47cb-af94-261dc14e28b0 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:46.938788 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-7a85156f-d852-47cb-af94-261dc14e28b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f803e64a0784410a2644431b378b664'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:47.046302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-7a85156f-d852-47cb-af94-261dc14e28b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f803e64a0784410a2644431b378b664'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:47.061499 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-7a85156f-d852-47cb-af94-261dc14e28b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f803e64a0784410a2644431b378b664'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:47.070988 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-7a85156f-d852-47cb-af94-261dc14e28b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f803e64a0784410a2644431b378b664'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:47.071765 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-7a85156f-d852-47cb-af94-261dc14e28b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f803e64a0784410a2644431b378b664'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:47.078752 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1197/2364] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:51:46 2022] DELETE /identity/v3/users/6f803e64a0784410a2644431b378b664 => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:51:47.088797 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d6bfa-6593-4353-9848-383c2236c843 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:47.089359 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d6bfa-6593-4353-9848-383c2236c843 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:47.089650 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d6bfa-6593-4353-9848-383c2236c843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:47.113377 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-5c3d6bfa-6593-4353-9848-383c2236c843 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:51:47.114611 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-5c3d6bfa-6593-4353-9848-383c2236c843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:47.182594 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:51:47.203283 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85156f-d852-47cb-af94-261dc14e28b0 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:47.203668 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1168/2365] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:51:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:51:47.206450 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-7a85156f-d852-47cb-af94-261dc14e28b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:47.250682 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:51:47.290142 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85156f-d852-47cb-af94-261dc14e28b0 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:47.296389 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2847c977-8069-4807-afb7-83bfff5695a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:47.297203 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847c977-8069-4807-afb7-83bfff5695a9 service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:47.297482 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847c977-8069-4807-afb7-83bfff5695a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:47.297757 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847c977-8069-4807-afb7-83bfff5695a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:47.308351 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2847c977-8069-4807-afb7-83bfff5695a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:47.309917 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2847c977-8069-4807-afb7-83bfff5695a9 service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:47.320177 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1198/2366] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:51:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:51:47.858922 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d6bfa-6593-4353-9848-383c2236c843 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:47.858922 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-5c3d6bfa-6593-4353-9848-383c2236c843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:47.888533 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d6bfa-6593-4353-9848-383c2236c843 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:47.894336 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8cb8a1-f9ca-4be1-abe6-865ce933f33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:47.895751 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8cb8a1-f9ca-4be1-abe6-865ce933f33f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:47.896311 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8cb8a1-f9ca-4be1-abe6-865ce933f33f admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:47.898494 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8cb8a1-f9ca-4be1-abe6-865ce933f33f admin admin] PATH_INFO: `/v3/projects/a0a9cfe3881945be8b9c12fa93c94fb9` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:47.900508 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8cb8a1-f9ca-4be1-abe6-865ce933f33f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0a9cfe3881945be8b9c12fa93c94fb9)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:47.904205 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8cb8a1-f9ca-4be1-abe6-865ce933f33f admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:48.039759 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eda93c-1a59-454f-89ca-3944a466c0c6 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:48.044145 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eda93c-1a59-454f-89ca-3944a466c0c6 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:48.044145 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eda93c-1a59-454f-89ca-3944a466c0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:48.060849 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-c5eda93c-1a59-454f-89ca-3944a466c0c6 None None] MFA Rules not processed for user `8cad1ee24ee84aa18da56d1145eac98a`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:51:48.062234 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-c5eda93c-1a59-454f-89ca-3944a466c0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:48.094148 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-0d8cb8a1-f9ca-4be1-abe6-865ce933f33f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0a9cfe3881945be8b9c12fa93c94fb9', 'request_id': 'req-0d8cb8a1-f9ca-4be1-abe6-865ce933f33f'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:48.123940 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1199/2367] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:51:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:51:48.192668 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-0d8cb8a1-f9ca-4be1-abe6-865ce933f33f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0a9cfe3881945be8b9c12fa93c94fb9', 'request_id': 'req-0d8cb8a1-f9ca-4be1-abe6-865ce933f33f'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:48.201391 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-0d8cb8a1-f9ca-4be1-abe6-865ce933f33f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0a9cfe3881945be8b9c12fa93c94fb9', 'request_id': 'req-0d8cb8a1-f9ca-4be1-abe6-865ce933f33f'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:48.219301 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-0d8cb8a1-f9ca-4be1-abe6-865ce933f33f admin admin] The token cache is being invalidate because project a0a9cfe3881945be8b9c12fa93c94fb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:51:48.219882 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-0d8cb8a1-f9ca-4be1-abe6-865ce933f33f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:48.220626 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-0d8cb8a1-f9ca-4be1-abe6-865ce933f33f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:48.222317 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1169/2368] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:51:47 2022] DELETE /identity/v3/projects/a0a9cfe3881945be8b9c12fa93c94fb9 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:51:48.226778 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eda93c-1a59-454f-89ca-3944a466c0c6 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:48.239591 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eda93c-1a59-454f-89ca-3944a466c0c6 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:48.337478 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3638d-7198-45ac-bf76-c5822c71f732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:48.338138 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3638d-7198-45ac-bf76-c5822c71f732 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:48.338426 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3638d-7198-45ac-bf76-c5822c71f732 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:48.338706 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3638d-7198-45ac-bf76-c5822c71f732 admin admin] PATH_INFO: `/v3/users/562282409632477f9513bf4dc3ad40e9` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:48.340401 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c3638d-7198-45ac-bf76-c5822c71f732 admin admin] RBAC: Authorizing `identity:delete_user(user_id=562282409632477f9513bf4dc3ad40e9)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:48.342500 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c3638d-7198-45ac-bf76-c5822c71f732 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:48.558773 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-26c3638d-7198-45ac-bf76-c5822c71f732 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '562282409632477f9513bf4dc3ad40e9'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:48.623171 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-26c3638d-7198-45ac-bf76-c5822c71f732 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '562282409632477f9513bf4dc3ad40e9'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:48.636838 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-26c3638d-7198-45ac-bf76-c5822c71f732 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '562282409632477f9513bf4dc3ad40e9'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:48.644804 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-26c3638d-7198-45ac-bf76-c5822c71f732 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '562282409632477f9513bf4dc3ad40e9'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:48.645603 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-26c3638d-7198-45ac-bf76-c5822c71f732 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '562282409632477f9513bf4dc3ad40e9'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:48.652206 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1200/2369] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:51:48 2022] DELETE /identity/v3/users/562282409632477f9513bf4dc3ad40e9 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:51:48.861389 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:51:49.264816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8cb8a1-f9ca-4be1-abe6-865ce933f33f admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:49.266679 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-0d8cb8a1-f9ca-4be1-abe6-865ce933f33f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:49.283738 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8cb8a1-f9ca-4be1-abe6-865ce933f33f admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:49.354373 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef3331d-c15a-4144-8288-726a8c17f87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:49.354928 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef3331d-c15a-4144-8288-726a8c17f87d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:49.355204 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef3331d-c15a-4144-8288-726a8c17f87d admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:49.355538 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef3331d-c15a-4144-8288-726a8c17f87d admin admin] PATH_INFO: `/v3/projects/ba87aa60bda44f9a8e4804bf6e010be4` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:49.356655 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef3331d-c15a-4144-8288-726a8c17f87d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba87aa60bda44f9a8e4804bf6e010be4)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:49.358622 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef3331d-c15a-4144-8288-726a8c17f87d admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:49.488782 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-aef3331d-c15a-4144-8288-726a8c17f87d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba87aa60bda44f9a8e4804bf6e010be4', 'request_id': 'req-aef3331d-c15a-4144-8288-726a8c17f87d'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:49.601498 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-aef3331d-c15a-4144-8288-726a8c17f87d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba87aa60bda44f9a8e4804bf6e010be4', 'request_id': 'req-aef3331d-c15a-4144-8288-726a8c17f87d'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:49.610975 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-aef3331d-c15a-4144-8288-726a8c17f87d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba87aa60bda44f9a8e4804bf6e010be4', 'request_id': 'req-aef3331d-c15a-4144-8288-726a8c17f87d'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:49.625513 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-aef3331d-c15a-4144-8288-726a8c17f87d admin admin] The token cache is being invalidate because project ba87aa60bda44f9a8e4804bf6e010be4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:51:49.625806 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-aef3331d-c15a-4144-8288-726a8c17f87d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:49.626838 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-aef3331d-c15a-4144-8288-726a8c17f87d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:49.628980 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1170/2370] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:51:49 2022] DELETE /identity/v3/projects/ba87aa60bda44f9a8e4804bf6e010be4 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:51:49.636192 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3638d-7198-45ac-bf76-c5822c71f732 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:49.637719 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-26c3638d-7198-45ac-bf76-c5822c71f732 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:49.658559 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3638d-7198-45ac-bf76-c5822c71f732 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:49.730601 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f409b7-4bea-4453-9cdd-a8d05b6d36c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:49.731254 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f409b7-4bea-4453-9cdd-a8d05b6d36c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:49.731532 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f409b7-4bea-4453-9cdd-a8d05b6d36c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:49.731806 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f409b7-4bea-4453-9cdd-a8d05b6d36c6 admin admin] PATH_INFO: `/v3/users/be48ce7718ff48c9b50c316e3e83a48f` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:49.732962 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f409b7-4bea-4453-9cdd-a8d05b6d36c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be48ce7718ff48c9b50c316e3e83a48f)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:49.734889 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f409b7-4bea-4453-9cdd-a8d05b6d36c6 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:49.942881 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-b7f409b7-4bea-4453-9cdd-a8d05b6d36c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be48ce7718ff48c9b50c316e3e83a48f'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:50.034691 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-b7f409b7-4bea-4453-9cdd-a8d05b6d36c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be48ce7718ff48c9b50c316e3e83a48f'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:50.049341 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-b7f409b7-4bea-4453-9cdd-a8d05b6d36c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be48ce7718ff48c9b50c316e3e83a48f'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:50.056916 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-b7f409b7-4bea-4453-9cdd-a8d05b6d36c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be48ce7718ff48c9b50c316e3e83a48f'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:50.057639 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-b7f409b7-4bea-4453-9cdd-a8d05b6d36c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be48ce7718ff48c9b50c316e3e83a48f'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:50.063594 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1201/2371] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:51:49 2022] DELETE /identity/v3/users/be48ce7718ff48c9b50c316e3e83a48f => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:51:50.582004 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef3331d-c15a-4144-8288-726a8c17f87d admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:50.583687 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-aef3331d-c15a-4144-8288-726a8c17f87d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:50.598997 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef3331d-c15a-4144-8288-726a8c17f87d admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:50.612769 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:51:50.670514 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dfbc9e-4415-4032-8182-4a6d98b06cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:50.671183 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfbc9e-4415-4032-8182-4a6d98b06cab admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:50.671459 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfbc9e-4415-4032-8182-4a6d98b06cab admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:50.671727 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfbc9e-4415-4032-8182-4a6d98b06cab admin admin] PATH_INFO: `/v3/projects/2227f20843d44004a35e39f7aa9d1f01` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:50.673464 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dfbc9e-4415-4032-8182-4a6d98b06cab admin admin] RBAC: Authorizing `identity:delete_project(project_id=2227f20843d44004a35e39f7aa9d1f01)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:50.675481 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dfbc9e-4415-4032-8182-4a6d98b06cab admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:50.798602 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-58dfbc9e-4415-4032-8182-4a6d98b06cab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2227f20843d44004a35e39f7aa9d1f01', 'request_id': 'req-58dfbc9e-4415-4032-8182-4a6d98b06cab'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:50.880041 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-58dfbc9e-4415-4032-8182-4a6d98b06cab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2227f20843d44004a35e39f7aa9d1f01', 'request_id': 'req-58dfbc9e-4415-4032-8182-4a6d98b06cab'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:50.886740 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-58dfbc9e-4415-4032-8182-4a6d98b06cab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2227f20843d44004a35e39f7aa9d1f01', 'request_id': 'req-58dfbc9e-4415-4032-8182-4a6d98b06cab'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:50.903498 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-58dfbc9e-4415-4032-8182-4a6d98b06cab admin admin] The token cache is being invalidate because project 2227f20843d44004a35e39f7aa9d1f01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:51:50.903796 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-58dfbc9e-4415-4032-8182-4a6d98b06cab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:50.904463 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-58dfbc9e-4415-4032-8182-4a6d98b06cab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:51:50.916570 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1171/2372] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:51:50 2022] DELETE /identity/v3/projects/2227f20843d44004a35e39f7aa9d1f01 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:51:50.921520 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426bc00a-5966-45ea-b562-f34115f6d1d2 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:50.922070 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426bc00a-5966-45ea-b562-f34115f6d1d2 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:50.922070 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426bc00a-5966-45ea-b562-f34115f6d1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:50.946427 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-426bc00a-5966-45ea-b562-f34115f6d1d2 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:51:50.947661 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-426bc00a-5966-45ea-b562-f34115f6d1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:51.015005 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1202/2373] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:51:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:51:51.023481 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b19709-b4fb-4609-8d5c-acbddb271c7b None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:51.023806 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b19709-b4fb-4609-8d5c-acbddb271c7b None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:51.024081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b19709-b4fb-4609-8d5c-acbddb271c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:51.045325 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-62b19709-b4fb-4609-8d5c-acbddb271c7b None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:51:51.046517 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-62b19709-b4fb-4609-8d5c-acbddb271c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:51.056568 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1172/2374] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:51:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:51:51.061094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426bc00a-5966-45ea-b562-f34115f6d1d2 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:51.071365 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426bc00a-5966-45ea-b562-f34115f6d1d2 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:51.076414 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7429df75-35e6-4fc0-9608-88651a713a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:51.077189 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429df75-35e6-4fc0-9608-88651a713a5d admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:51.077471 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429df75-35e6-4fc0-9608-88651a713a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:51.077746 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429df75-35e6-4fc0-9608-88651a713a5d admin admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:51.078562 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7429df75-35e6-4fc0-9608-88651a713a5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:51:51.078897 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7429df75-35e6-4fc0-9608-88651a713a5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:51.080545 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7429df75-35e6-4fc0-9608-88651a713a5d admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:51.088296 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:51:51.095280 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1203/2375] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:51:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:51:51.097713 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b19709-b4fb-4609-8d5c-acbddb271c7b None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:51.107784 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b19709-b4fb-4609-8d5c-acbddb271c7b None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:51.112807 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec1b06-52b2-4fe6-848d-23b193f6c080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:51.113415 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec1b06-52b2-4fe6-848d-23b193f6c080 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:51.113694 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec1b06-52b2-4fe6-848d-23b193f6c080 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:51.113967 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec1b06-52b2-4fe6-848d-23b193f6c080 admin admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:51.114611 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ec1b06-52b2-4fe6-848d-23b193f6c080 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:51.117807 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ec1b06-52b2-4fe6-848d-23b193f6c080 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:51.262847 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1173/2376] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:51:51 2022] POST /identity/v3/projects => generated 383 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:51:51.267247 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7429df75-35e6-4fc0-9608-88651a713a5d admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:51.279006 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7429df75-35e6-4fc0-9608-88651a713a5d admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:51.338597 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab62cba-b7c3-4a85-b943-5ea7713192bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:51.339216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab62cba-b7c3-4a85-b943-5ea7713192bc admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:51.339625 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab62cba-b7c3-4a85-b943-5ea7713192bc admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:51.339909 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab62cba-b7c3-4a85-b943-5ea7713192bc admin admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:51.340653 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab62cba-b7c3-4a85-b943-5ea7713192bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:51.342651 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab62cba-b7c3-4a85-b943-5ea7713192bc admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:51.416442 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1204/2377] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:51:51 2022] POST /identity/v3/users => generated 292 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:51:51.420658 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec1b06-52b2-4fe6-848d-23b193f6c080 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:51.430803 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec1b06-52b2-4fe6-848d-23b193f6c080 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:51.435442 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c896ec0-0e1d-4aa3-ba68-c8f704d49a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:51.436029 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c896ec0-0e1d-4aa3-ba68-c8f704d49a48 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:51.436307 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c896ec0-0e1d-4aa3-ba68-c8f704d49a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:51.436578 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c896ec0-0e1d-4aa3-ba68-c8f704d49a48 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:51.437252 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c896ec0-0e1d-4aa3-ba68-c8f704d49a48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:51.438838 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c896ec0-0e1d-4aa3-ba68-c8f704d49a48 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:51.448737 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1174/2378] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:51:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:51:51.452755 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab62cba-b7c3-4a85-b943-5ea7713192bc admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:51.464450 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab62cba-b7c3-4a85-b943-5ea7713192bc admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:51.469141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ba623-ad65-4ba5-8fa7-c9bb67a28e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:51.469737 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ba623-ad65-4ba5-8fa7-c9bb67a28e79 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:51.470014 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ba623-ad65-4ba5-8fa7-c9bb67a28e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:51.470288 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ba623-ad65-4ba5-8fa7-c9bb67a28e79 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:51.470928 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9ba623-ad65-4ba5-8fa7-c9bb67a28e79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:51.472517 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9ba623-ad65-4ba5-8fa7-c9bb67a28e79 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:51.484354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1205/2379] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:51:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:51:51.488677 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c896ec0-0e1d-4aa3-ba68-c8f704d49a48 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:51.499186 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c896ec0-0e1d-4aa3-ba68-c8f704d49a48 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:51.503925 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b822728-2746-42bb-ad87-57150755d9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:51.504856 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b822728-2746-42bb-ad87-57150755d9b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:51.512147 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b822728-2746-42bb-ad87-57150755d9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:51.512147 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b822728-2746-42bb-ad87-57150755d9b8 admin admin] PATH_INFO: `/v3/projects/a8c824e8723341808544a577b85d9b2d/users/70cf6f471ff04dab9f8cbc268397928e/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:51.523389 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b822728-2746-42bb-ad87-57150755d9b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8c824e8723341808544a577b85d9b2d, user_id=70cf6f471ff04dab9f8cbc268397928e, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:51.526083 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b822728-2746-42bb-ad87-57150755d9b8 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:51.689883 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1175/2380] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:51:51 2022] PUT /identity/v3/projects/a8c824e8723341808544a577b85d9b2d/users/70cf6f471ff04dab9f8cbc268397928e/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:51:51.695592 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6834ab-ef0c-4f64-ae36-dec9cc0753b7 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:51.695879 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6834ab-ef0c-4f64-ae36-dec9cc0753b7 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:51.696158 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6834ab-ef0c-4f64-ae36-dec9cc0753b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:51.716187 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-eb6834ab-ef0c-4f64-ae36-dec9cc0753b7 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:51:51.717421 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-eb6834ab-ef0c-4f64-ae36-dec9cc0753b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:51.776712 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1206/2381] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:51:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:51:51.786361 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b822728-2746-42bb-ad87-57150755d9b8 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:51.788172 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-3b822728-2746-42bb-ad87-57150755d9b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:51.831133 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:51:51.846262 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b822728-2746-42bb-ad87-57150755d9b8 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:51.850210 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5454301-500a-49dd-a3b1-15a25ed1d35b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:51.850828 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5454301-500a-49dd-a3b1-15a25ed1d35b service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:51.851108 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5454301-500a-49dd-a3b1-15a25ed1d35b service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:51.857919 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5454301-500a-49dd-a3b1-15a25ed1d35b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:51.860956 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5454301-500a-49dd-a3b1-15a25ed1d35b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:51.862383 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5454301-500a-49dd-a3b1-15a25ed1d35b service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:51.872314 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1176/2382] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:51:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:51:53.378483 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2184306-f66a-4fa4-8c36-1c1829b0cd10 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:53.379127 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2184306-f66a-4fa4-8c36-1c1829b0cd10 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:53.379127 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2184306-f66a-4fa4-8c36-1c1829b0cd10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:53.417270 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-f2184306-f66a-4fa4-8c36-1c1829b0cd10 None None] MFA Rules not processed for user `70cf6f471ff04dab9f8cbc268397928e`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:51:53.418487 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-f2184306-f66a-4fa4-8c36-1c1829b0cd10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:53.487977 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1207/2383] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:51:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:51:53.500455 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5454301-500a-49dd-a3b1-15a25ed1d35b service neutron] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:53.503045 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-b5454301-500a-49dd-a3b1-15a25ed1d35b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:53.568926 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5454301-500a-49dd-a3b1-15a25ed1d35b service neutron] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:53.573304 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c0edb-cfcd-4f18-b705-46a49c61adbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:53.573973 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c0edb-cfcd-4f18-b705-46a49c61adbe service cinder] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:53.574158 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c0edb-cfcd-4f18-b705-46a49c61adbe service cinder] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:53.574576 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c0edb-cfcd-4f18-b705-46a49c61adbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:53.583453 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8c0edb-cfcd-4f18-b705-46a49c61adbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:53.585001 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8c0edb-cfcd-4f18-b705-46a49c61adbe service cinder] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:53.595353 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1177/2384] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:51:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:51:55.057815 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ade601-8f9c-488f-ab0d-df92e2d14104 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:55.058302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ade601-8f9c-488f-ab0d-df92e2d14104 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:55.058391 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa70aa1-8c64-49b7-8a61-4b047ad78b4d None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:55.058732 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ade601-8f9c-488f-ab0d-df92e2d14104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:55.058848 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa70aa1-8c64-49b7-8a61-4b047ad78b4d None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:55.058848 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa70aa1-8c64-49b7-8a61-4b047ad78b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:55.072708 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:51:55.078524 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-3fa70aa1-8c64-49b7-8a61-4b047ad78b4d None None] MFA Rules not processed for user `f77dfe78b1a8405084c06d9acee96511`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:51:55.079422 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-43ade601-8f9c-488f-ab0d-df92e2d14104 None None] MFA Rules not processed for user `f77dfe78b1a8405084c06d9acee96511`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:51:55.079753 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-3fa70aa1-8c64-49b7-8a61-4b047ad78b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:55.080598 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-43ade601-8f9c-488f-ab0d-df92e2d14104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:55.140060 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1178/2385] 10.222.0.52 () {58 vars in 1009 bytes} [Mon Aug 29 22:51:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:51:55.140837 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1208/2386] 10.222.0.52 () {58 vars in 1009 bytes} [Mon Aug 29 22:51:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:51:55.155313 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa70aa1-8c64-49b7-8a61-4b047ad78b4d None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:55.162616 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-3fa70aa1-8c64-49b7-8a61-4b047ad78b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:55.165495 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ade601-8f9c-488f-ab0d-df92e2d14104 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:55.167254 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-43ade601-8f9c-488f-ab0d-df92e2d14104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:55.252103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa70aa1-8c64-49b7-8a61-4b047ad78b4d None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:55.264362 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa84dc8b-f9c4-4547-b00c-807717bbb722 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57c175ac90e545358746176972ff39a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:55.264362 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa84dc8b-f9c4-4547-b00c-807717bbb722 service swift] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:55.264362 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa84dc8b-f9c4-4547-b00c-807717bbb722 service swift] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:55.264362 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa84dc8b-f9c4-4547-b00c-807717bbb722 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:55.274496 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa84dc8b-f9c4-4547-b00c-807717bbb722 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:55.276303 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa84dc8b-f9c4-4547-b00c-807717bbb722 service swift] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:55.279008 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ade601-8f9c-488f-ab0d-df92e2d14104 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:55.286016 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a6cbf3-ab08-4da2-9a3c-26e819a62c34 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57c175ac90e545358746176972ff39a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:55.286490 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1179/2387] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Aug 29 22:51:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 22:51:55.286713 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6cbf3-ab08-4da2-9a3c-26e819a62c34 service swift] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:55.286995 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6cbf3-ab08-4da2-9a3c-26e819a62c34 service swift] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:55.298312 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6cbf3-ab08-4da2-9a3c-26e819a62c34 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:55.298312 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a6cbf3-ab08-4da2-9a3c-26e819a62c34 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:55.301013 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a6cbf3-ab08-4da2-9a3c-26e819a62c34 service swift] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:55.312011 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1209/2388] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Aug 29 22:51:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 29 22:51:55.388556 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e754d4-5176-48fe-8aed-e40f1af39886 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:55.388874 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e754d4-5176-48fe-8aed-e40f1af39886 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:55.389149 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e754d4-5176-48fe-8aed-e40f1af39886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:55.412452 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-29e754d4-5176-48fe-8aed-e40f1af39886 None None] MFA Rules not processed for user `70cf6f471ff04dab9f8cbc268397928e`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:51:55.414475 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-29e754d4-5176-48fe-8aed-e40f1af39886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:55.426259 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1180/2389] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:51:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:52:05.351483 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=54 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:52:05.404672 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=94 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:52:14.609799 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a59037-34c9-4920-bf5d-ebe03a682040 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:52:14.610319 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a59037-34c9-4920-bf5d-ebe03a682040 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:52:14.610424 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a59037-34c9-4920-bf5d-ebe03a682040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:52:14.630283 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-f1a59037-34c9-4920-bf5d-ebe03a682040 None None] MFA Rules not processed for user `f77dfe78b1a8405084c06d9acee96511`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:52:14.631543 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-f1a59037-34c9-4920-bf5d-ebe03a682040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:52:14.639227 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1210/2390] 10.222.0.52 () {58 vars in 967 bytes} [Mon Aug 29 22:52:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:52:14.652849 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e754d4-5176-48fe-8aed-e40f1af39886 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:52:14.667802 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e754d4-5176-48fe-8aed-e40f1af39886 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:52:14.670837 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0247140-fbcb-4686-9627-ae705fdb2005 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57c175ac90e545358746176972ff39a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:52:14.674839 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0247140-fbcb-4686-9627-ae705fdb2005 service swift] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:52:14.674839 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0247140-fbcb-4686-9627-ae705fdb2005 service swift] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:52:14.674839 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0247140-fbcb-4686-9627-ae705fdb2005 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:52:14.686571 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0247140-fbcb-4686-9627-ae705fdb2005 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:52:14.688197 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0247140-fbcb-4686-9627-ae705fdb2005 service swift] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:52:14.696834 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1181/2391] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Aug 29 22:52:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 22:52:18.034184 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a59037-34c9-4920-bf5d-ebe03a682040 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:52:18.035794 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-f1a59037-34c9-4920-bf5d-ebe03a682040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:52:18.055777 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a59037-34c9-4920-bf5d-ebe03a682040 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:52:18.059667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b25f6b7-7033-4867-9356-593cccfa412c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:52:18.060322 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b25f6b7-7033-4867-9356-593cccfa412c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:52:18.060598 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b25f6b7-7033-4867-9356-593cccfa412c admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:52:18.060905 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b25f6b7-7033-4867-9356-593cccfa412c admin admin] PATH_INFO: `/v3/users/9c3c51d98e274150933a1306f7747f42` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:52:18.062136 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b25f6b7-7033-4867-9356-593cccfa412c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c3c51d98e274150933a1306f7747f42)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:52:18.064027 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b25f6b7-7033-4867-9356-593cccfa412c admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:52:18.329140 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-3b25f6b7-7033-4867-9356-593cccfa412c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c3c51d98e274150933a1306f7747f42'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:52:18.400310 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-3b25f6b7-7033-4867-9356-593cccfa412c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c3c51d98e274150933a1306f7747f42'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:52:18.413071 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-3b25f6b7-7033-4867-9356-593cccfa412c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c3c51d98e274150933a1306f7747f42'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:52:18.420998 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-3b25f6b7-7033-4867-9356-593cccfa412c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c3c51d98e274150933a1306f7747f42'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:52:18.421690 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-3b25f6b7-7033-4867-9356-593cccfa412c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c3c51d98e274150933a1306f7747f42'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:52:18.427522 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1211/2392] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:52:18 2022] DELETE /identity/v3/users/9c3c51d98e274150933a1306f7747f42 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:52:18.790702 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0247140-fbcb-4686-9627-ae705fdb2005 service swift] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:52:18.803214 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0247140-fbcb-4686-9627-ae705fdb2005 service swift] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:52:18.874309 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a8e3f-e399-4220-8787-c6e2f27fdbc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:52:18.875123 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a8e3f-e399-4220-8787-c6e2f27fdbc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:52:18.875287 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a8e3f-e399-4220-8787-c6e2f27fdbc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:52:18.875570 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a8e3f-e399-4220-8787-c6e2f27fdbc7 admin admin] PATH_INFO: `/v3/projects/4f38e29d63fd4d04984af578b1708e4a` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:52:18.876860 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919a8e3f-e399-4220-8787-c6e2f27fdbc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f38e29d63fd4d04984af578b1708e4a)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:52:18.878816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919a8e3f-e399-4220-8787-c6e2f27fdbc7 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:52:19.042978 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-919a8e3f-e399-4220-8787-c6e2f27fdbc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f38e29d63fd4d04984af578b1708e4a', 'request_id': 'req-919a8e3f-e399-4220-8787-c6e2f27fdbc7'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:52:19.149250 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-919a8e3f-e399-4220-8787-c6e2f27fdbc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f38e29d63fd4d04984af578b1708e4a', 'request_id': 'req-919a8e3f-e399-4220-8787-c6e2f27fdbc7'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:52:19.160480 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-919a8e3f-e399-4220-8787-c6e2f27fdbc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f38e29d63fd4d04984af578b1708e4a', 'request_id': 'req-919a8e3f-e399-4220-8787-c6e2f27fdbc7'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:52:19.168896 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-919a8e3f-e399-4220-8787-c6e2f27fdbc7 admin admin] The token cache is being invalidate because project 4f38e29d63fd4d04984af578b1708e4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:52:19.169188 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-919a8e3f-e399-4220-8787-c6e2f27fdbc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:52:19.170047 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-919a8e3f-e399-4220-8787-c6e2f27fdbc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:52:19.171935 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1182/2393] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:52:18 2022] DELETE /identity/v3/projects/4f38e29d63fd4d04984af578b1708e4a => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:52:19.177043 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46150d59-78ed-41bd-a69c-0889482d5916 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:52:19.183833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46150d59-78ed-41bd-a69c-0889482d5916 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:52:19.183833 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46150d59-78ed-41bd-a69c-0889482d5916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:52:19.199056 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-46150d59-78ed-41bd-a69c-0889482d5916 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:52:19.200271 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-46150d59-78ed-41bd-a69c-0889482d5916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:52:19.265878 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1212/2394] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:52:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:52:19.276138 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520eb30-dd67-4cdd-b712-d716a4b199e9 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:52:19.287197 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520eb30-dd67-4cdd-b712-d716a4b199e9 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:52:19.287197 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520eb30-dd67-4cdd-b712-d716a4b199e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:52:19.297978 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-8520eb30-dd67-4cdd-b712-d716a4b199e9 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:52:19.299603 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-8520eb30-dd67-4cdd-b712-d716a4b199e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:52:19.307953 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1183/2395] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:52:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:52:19.312338 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46150d59-78ed-41bd-a69c-0889482d5916 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:52:19.322780 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46150d59-78ed-41bd-a69c-0889482d5916 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:52:19.327757 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe679560-9884-4132-b6d0-84532eaaebb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:52:19.328612 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe679560-9884-4132-b6d0-84532eaaebb3 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:52:19.328612 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe679560-9884-4132-b6d0-84532eaaebb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:52:19.328935 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe679560-9884-4132-b6d0-84532eaaebb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:52:19.329607 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe679560-9884-4132-b6d0-84532eaaebb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:52:19.329965 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe679560-9884-4132-b6d0-84532eaaebb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:52:19.331565 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe679560-9884-4132-b6d0-84532eaaebb3 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:52:19.344830 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1213/2396] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:52:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:52:19.349710 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8520eb30-dd67-4cdd-b712-d716a4b199e9 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:52:19.360420 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8520eb30-dd67-4cdd-b712-d716a4b199e9 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:52:19.370093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962b7343-1501-4846-95c4-705640714a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:52:19.370093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b7343-1501-4846-95c4-705640714a34 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:52:19.370093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b7343-1501-4846-95c4-705640714a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:52:19.370093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b7343-1501-4846-95c4-705640714a34 admin admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:52:19.370564 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962b7343-1501-4846-95c4-705640714a34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:52:19.370871 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962b7343-1501-4846-95c4-705640714a34 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:52:19.524916 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1184/2397] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:52:19 2022] POST /identity/v3/projects => generated 365 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:52:19.526097 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe679560-9884-4132-b6d0-84532eaaebb3 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:52:19.536503 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe679560-9884-4132-b6d0-84532eaaebb3 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:52:19.594538 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:52:19.605569 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5cf609-1939-4f12-b3a8-91bb89247a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:52:19.606469 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5cf609-1939-4f12-b3a8-91bb89247a05 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:52:19.606821 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5cf609-1939-4f12-b3a8-91bb89247a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:52:19.607186 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5cf609-1939-4f12-b3a8-91bb89247a05 admin admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:52:19.608290 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5cf609-1939-4f12-b3a8-91bb89247a05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:52:19.610065 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5cf609-1939-4f12-b3a8-91bb89247a05 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:52:19.720782 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1214/2398] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:52:19 2022] POST /identity/v3/users => generated 283 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:52:19.725211 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962b7343-1501-4846-95c4-705640714a34 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:52:19.737766 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962b7343-1501-4846-95c4-705640714a34 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:52:19.743333 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28e7984-a401-4fad-b079-da8003917219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:52:19.743980 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e7984-a401-4fad-b079-da8003917219 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:52:19.744255 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e7984-a401-4fad-b079-da8003917219 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:52:19.744559 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e7984-a401-4fad-b079-da8003917219 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:52:19.745306 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28e7984-a401-4fad-b079-da8003917219 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:52:19.747075 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28e7984-a401-4fad-b079-da8003917219 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:52:19.758049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1185/2399] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:52:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:52:19.764631 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5cf609-1939-4f12-b3a8-91bb89247a05 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:52:19.774924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5cf609-1939-4f12-b3a8-91bb89247a05 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:52:19.779504 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07e3a67-78d2-4e73-b95c-35afffd8d640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:52:19.780094 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e3a67-78d2-4e73-b95c-35afffd8d640 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:52:19.780371 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e3a67-78d2-4e73-b95c-35afffd8d640 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:52:19.780643 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e3a67-78d2-4e73-b95c-35afffd8d640 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:52:19.781313 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07e3a67-78d2-4e73-b95c-35afffd8d640 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:52:19.782888 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07e3a67-78d2-4e73-b95c-35afffd8d640 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:52:19.794070 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1215/2400] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:52:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:52:19.798273 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28e7984-a401-4fad-b079-da8003917219 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:52:19.808844 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28e7984-a401-4fad-b079-da8003917219 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:52:19.813450 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4035f934-5e4f-432a-ba0f-5202d73e941c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:52:19.814094 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035f934-5e4f-432a-ba0f-5202d73e941c admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:52:19.814372 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035f934-5e4f-432a-ba0f-5202d73e941c admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:52:19.814648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035f934-5e4f-432a-ba0f-5202d73e941c admin admin] PATH_INFO: `/v3/projects/56b2e82c095b4b01887da5a8a5c2e9d8/users/520da287d35b4e52840cdf972eecf551/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:52:19.829807 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4035f934-5e4f-432a-ba0f-5202d73e941c admin admin] RBAC: Authorizing `identity:create_grant(project_id=56b2e82c095b4b01887da5a8a5c2e9d8, user_id=520da287d35b4e52840cdf972eecf551, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:52:19.837493 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4035f934-5e4f-432a-ba0f-5202d73e941c admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:52:19.947644 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1186/2401] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:52:19 2022] PUT /identity/v3/projects/56b2e82c095b4b01887da5a8a5c2e9d8/users/520da287d35b4e52840cdf972eecf551/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:52:19.959066 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e4bd6-d394-49f5-845e-4b80afa1cadf None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:52:19.962243 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e4bd6-d394-49f5-845e-4b80afa1cadf None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:52:19.962243 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e4bd6-d394-49f5-845e-4b80afa1cadf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:52:19.992979 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-9f2e4bd6-d394-49f5-845e-4b80afa1cadf None None] MFA Rules not processed for user `520da287d35b4e52840cdf972eecf551`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:52:19.992979 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-9f2e4bd6-d394-49f5-845e-4b80afa1cadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:52:20.060289 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1216/2402] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:52:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:52:20.072325 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4035f934-5e4f-432a-ba0f-5202d73e941c admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:52:20.074027 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-4035f934-5e4f-432a-ba0f-5202d73e941c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:52:20.090215 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:52:20.140497 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4035f934-5e4f-432a-ba0f-5202d73e941c admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:52:20.145613 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c58df-9578-4616-80a0-30ab27264aaf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c2569e7cd774199aa5750fd6a850060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:52:20.146145 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c58df-9578-4616-80a0-30ab27264aaf service glance] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:52:20.149377 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c58df-9578-4616-80a0-30ab27264aaf service glance] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:52:20.149680 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c58df-9578-4616-80a0-30ab27264aaf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:52:20.169401 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c58df-9578-4616-80a0-30ab27264aaf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:52:20.170936 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c58df-9578-4616-80a0-30ab27264aaf service glance] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:52:20.182524 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1187/2403] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:52:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:52:28.997782 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2e4bd6-d394-49f5-845e-4b80afa1cadf None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:52:28.999230 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-9f2e4bd6-d394-49f5-845e-4b80afa1cadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:52:29.063526 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2e4bd6-d394-49f5-845e-4b80afa1cadf None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:52:29.067117 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74746054-4ba9-416d-ae8d-978a1a59cf2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:52:29.067760 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74746054-4ba9-416d-ae8d-978a1a59cf2c service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:52:29.068048 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74746054-4ba9-416d-ae8d-978a1a59cf2c service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:52:29.068329 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74746054-4ba9-416d-ae8d-978a1a59cf2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:52:29.070223 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-74746054-4ba9-416d-ae8d-978a1a59cf2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:52:29.140441 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74746054-4ba9-416d-ae8d-978a1a59cf2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:52:29.141919 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74746054-4ba9-416d-ae8d-978a1a59cf2c service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:52:29.152280 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1217/2404] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:52:28 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:52:31.252279 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=20 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:52:39.147127 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:52:40.209375 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b551718-640b-4bf9-9309-8f6e245565d5 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:52:40.209924 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b551718-640b-4bf9-9309-8f6e245565d5 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:52:40.209924 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b551718-640b-4bf9-9309-8f6e245565d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:52:40.229676 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-1b551718-640b-4bf9-9309-8f6e245565d5 None None] MFA Rules not processed for user `38a89946e1fb4b43aed992078dba0b4f`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:52:40.230883 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-1b551718-640b-4bf9-9309-8f6e245565d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:52:40.298085 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1188/2405] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:52:40 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:52:41.058480 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960019fa-6f54-4ef4-a289-0baf3a95b708 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:52:41.059021 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960019fa-6f54-4ef4-a289-0baf3a95b708 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:52:41.059021 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960019fa-6f54-4ef4-a289-0baf3a95b708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:52:41.078887 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-960019fa-6f54-4ef4-a289-0baf3a95b708 None None] MFA Rules not processed for user `53f9f8eb7a20431ca0d679af45946bce`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:52:41.080100 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-960019fa-6f54-4ef4-a289-0baf3a95b708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:52:41.140447 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1218/2406] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:52:41 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:52:50.283567 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=25 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:52:51.133329 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=25 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:52:56.436098 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b551718-640b-4bf9-9309-8f6e245565d5 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:52:56.501076 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b551718-640b-4bf9-9309-8f6e245565d5 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:52:56.505573 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf95df-5a4a-4225-9da6-968a15c0c243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:52:56.506195 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf95df-5a4a-4225-9da6-968a15c0c243 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:52:56.506474 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf95df-5a4a-4225-9da6-968a15c0c243 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:52:56.506747 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf95df-5a4a-4225-9da6-968a15c0c243 admin admin] PATH_INFO: `/v3/users/520da287d35b4e52840cdf972eecf551` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:52:56.507858 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadf95df-5a4a-4225-9da6-968a15c0c243 admin admin] RBAC: Authorizing `identity:delete_user(user_id=520da287d35b4e52840cdf972eecf551)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:52:56.510440 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadf95df-5a4a-4225-9da6-968a15c0c243 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:52:56.762710 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-dadf95df-5a4a-4225-9da6-968a15c0c243 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '520da287d35b4e52840cdf972eecf551'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:52:56.777651 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-dadf95df-5a4a-4225-9da6-968a15c0c243 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '520da287d35b4e52840cdf972eecf551'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:52:56.786126 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-dadf95df-5a4a-4225-9da6-968a15c0c243 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '520da287d35b4e52840cdf972eecf551'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:52:56.914211 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-dadf95df-5a4a-4225-9da6-968a15c0c243 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '520da287d35b4e52840cdf972eecf551'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:52:56.914910 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-dadf95df-5a4a-4225-9da6-968a15c0c243 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '520da287d35b4e52840cdf972eecf551'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:52:56.921922 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1189/2407] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:52:56 2022] DELETE /identity/v3/users/520da287d35b4e52840cdf972eecf551 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:52:56.926451 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0979da7-d3a4-4e23-af10-bb9cb8174e2c None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:52:56.926955 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0979da7-d3a4-4e23-af10-bb9cb8174e2c None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:52:56.927099 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0979da7-d3a4-4e23-af10-bb9cb8174e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:52:56.946123 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-c0979da7-d3a4-4e23-af10-bb9cb8174e2c None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:52:56.948043 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-c0979da7-d3a4-4e23-af10-bb9cb8174e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:52:57.024819 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1219/2408] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:52:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:52:57.026797 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf95df-5a4a-4225-9da6-968a15c0c243 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:52:57.028245 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-dadf95df-5a4a-4225-9da6-968a15c0c243 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:52:57.089562 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf95df-5a4a-4225-9da6-968a15c0c243 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:52:57.093318 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de506e87-1f0b-402b-8eff-6bc3e5897274 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:52:57.094020 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de506e87-1f0b-402b-8eff-6bc3e5897274 service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:52:57.094297 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de506e87-1f0b-402b-8eff-6bc3e5897274 service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:52:57.094569 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de506e87-1f0b-402b-8eff-6bc3e5897274 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:52:57.104064 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de506e87-1f0b-402b-8eff-6bc3e5897274 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:52:57.105540 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de506e87-1f0b-402b-8eff-6bc3e5897274 service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:52:57.116452 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1190/2409] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:52:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:52:57.711035 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0979da7-d3a4-4e23-af10-bb9cb8174e2c None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:52:57.712823 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-c0979da7-d3a4-4e23-af10-bb9cb8174e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:52:57.728118 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0979da7-d3a4-4e23-af10-bb9cb8174e2c None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:52:57.743905 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2078000-0e60-43a6-9bd4-00b18e189dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:52:57.744598 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2078000-0e60-43a6-9bd4-00b18e189dfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:52:57.744918 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2078000-0e60-43a6-9bd4-00b18e189dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:52:57.745314 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2078000-0e60-43a6-9bd4-00b18e189dfb admin admin] PATH_INFO: `/v3/projects/56b2e82c095b4b01887da5a8a5c2e9d8` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:52:57.746353 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2078000-0e60-43a6-9bd4-00b18e189dfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=56b2e82c095b4b01887da5a8a5c2e9d8)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:52:57.749209 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2078000-0e60-43a6-9bd4-00b18e189dfb admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:52:57.903265 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-e2078000-0e60-43a6-9bd4-00b18e189dfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56b2e82c095b4b01887da5a8a5c2e9d8', 'request_id': 'req-e2078000-0e60-43a6-9bd4-00b18e189dfb'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:52:57.909184 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-e2078000-0e60-43a6-9bd4-00b18e189dfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56b2e82c095b4b01887da5a8a5c2e9d8', 'request_id': 'req-e2078000-0e60-43a6-9bd4-00b18e189dfb'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:52:57.915649 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-e2078000-0e60-43a6-9bd4-00b18e189dfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56b2e82c095b4b01887da5a8a5c2e9d8', 'request_id': 'req-e2078000-0e60-43a6-9bd4-00b18e189dfb'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:52:58.017220 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-e2078000-0e60-43a6-9bd4-00b18e189dfb admin admin] The token cache is being invalidate because project 56b2e82c095b4b01887da5a8a5c2e9d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:52:58.017838 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-e2078000-0e60-43a6-9bd4-00b18e189dfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:52:58.018101 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-e2078000-0e60-43a6-9bd4-00b18e189dfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:52:58.019621 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1220/2410] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:52:57 2022] DELETE /identity/v3/projects/56b2e82c095b4b01887da5a8a5c2e9d8 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:52:58.024529 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ffe176-6d86-4595-8335-869bcecad11a None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:52:58.025055 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ffe176-6d86-4595-8335-869bcecad11a None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:52:58.025142 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ffe176-6d86-4595-8335-869bcecad11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:52:58.053302 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-f4ffe176-6d86-4595-8335-869bcecad11a None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:52:58.054704 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-f4ffe176-6d86-4595-8335-869bcecad11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:52:58.076521 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:52:58.143687 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1191/2411] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:52:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:52:58.151930 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea6e2c7-87a3-48d4-bd81-789827a302ac None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:52:58.152214 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea6e2c7-87a3-48d4-bd81-789827a302ac None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:52:58.152495 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea6e2c7-87a3-48d4-bd81-789827a302ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:52:58.172696 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-1ea6e2c7-87a3-48d4-bd81-789827a302ac None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:52:58.173944 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-1ea6e2c7-87a3-48d4-bd81-789827a302ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:52:58.183307 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1221/2412] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:52:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:52:58.187822 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ffe176-6d86-4595-8335-869bcecad11a None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:52:58.201567 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ffe176-6d86-4595-8335-869bcecad11a None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:52:58.207814 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0ffaa-36ea-4908-a1a9-b49d7461e82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:52:58.208648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0ffaa-36ea-4908-a1a9-b49d7461e82f admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:52:58.209117 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0ffaa-36ea-4908-a1a9-b49d7461e82f admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:52:58.209462 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0ffaa-36ea-4908-a1a9-b49d7461e82f admin admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:52:58.210502 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb0ffaa-36ea-4908-a1a9-b49d7461e82f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:52:58.211041 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb0ffaa-36ea-4908-a1a9-b49d7461e82f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:52:58.212653 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb0ffaa-36ea-4908-a1a9-b49d7461e82f admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:52:58.225419 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1192/2413] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:52:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:52:58.230328 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea6e2c7-87a3-48d4-bd81-789827a302ac None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:52:58.240266 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea6e2c7-87a3-48d4-bd81-789827a302ac None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:52:58.246051 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4e0979-1b39-4cb8-b62c-079a2c11280b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:52:58.246880 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e0979-1b39-4cb8-b62c-079a2c11280b admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:52:58.247159 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e0979-1b39-4cb8-b62c-079a2c11280b admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:52:58.247434 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e0979-1b39-4cb8-b62c-079a2c11280b admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:52:58.248066 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4e0979-1b39-4cb8-b62c-079a2c11280b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:52:58.250152 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4e0979-1b39-4cb8-b62c-079a2c11280b admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:52:58.360415 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1222/2414] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:52:58 2022] POST /identity/v3/projects => generated 373 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:52:58.361678 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0ffaa-36ea-4908-a1a9-b49d7461e82f admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:52:58.372880 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0ffaa-36ea-4908-a1a9-b49d7461e82f admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:52:58.432808 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec7e159-fd9c-43ad-9cdc-6f0d7d084972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:52:58.433615 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec7e159-fd9c-43ad-9cdc-6f0d7d084972 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:52:58.433741 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec7e159-fd9c-43ad-9cdc-6f0d7d084972 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:52:58.434097 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec7e159-fd9c-43ad-9cdc-6f0d7d084972 admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:52:58.434573 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec7e159-fd9c-43ad-9cdc-6f0d7d084972 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:52:58.436499 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec7e159-fd9c-43ad-9cdc-6f0d7d084972 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:52:58.536576 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1193/2415] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:52:58 2022] POST /identity/v3/users => generated 287 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:52:58.541559 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4e0979-1b39-4cb8-b62c-079a2c11280b admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:52:58.553401 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4e0979-1b39-4cb8-b62c-079a2c11280b admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:52:58.559033 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1f9089-30c6-45de-a179-dd11804ed7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:52:58.559726 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f9089-30c6-45de-a179-dd11804ed7af admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:52:58.560009 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f9089-30c6-45de-a179-dd11804ed7af admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:52:58.560283 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f9089-30c6-45de-a179-dd11804ed7af admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:52:58.560967 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1f9089-30c6-45de-a179-dd11804ed7af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:52:58.562585 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1f9089-30c6-45de-a179-dd11804ed7af admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:52:58.573158 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1223/2416] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:52:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:52:58.577264 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec7e159-fd9c-43ad-9cdc-6f0d7d084972 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:52:58.588588 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec7e159-fd9c-43ad-9cdc-6f0d7d084972 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:52:58.593238 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eae6e0d-eb9c-4a27-a60f-f7e7b37179a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:52:58.593860 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae6e0d-eb9c-4a27-a60f-f7e7b37179a4 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:52:58.594144 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae6e0d-eb9c-4a27-a60f-f7e7b37179a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:52:58.594416 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae6e0d-eb9c-4a27-a60f-f7e7b37179a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:52:58.595076 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eae6e0d-eb9c-4a27-a60f-f7e7b37179a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:52:58.596690 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eae6e0d-eb9c-4a27-a60f-f7e7b37179a4 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:52:58.612885 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1194/2417] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:52:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:52:58.613878 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1f9089-30c6-45de-a179-dd11804ed7af admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:52:58.625647 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1f9089-30c6-45de-a179-dd11804ed7af admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:52:58.632931 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db417b11-6ec0-4f45-a87c-8999584d1faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:52:58.634049 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db417b11-6ec0-4f45-a87c-8999584d1faa admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:52:58.634165 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db417b11-6ec0-4f45-a87c-8999584d1faa admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:52:58.634472 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db417b11-6ec0-4f45-a87c-8999584d1faa admin admin] PATH_INFO: `/v3/projects/3dacdd3e7b3240aabacdf5c385d5b9b2/users/785d6ff244c841df97f5b3cbfc56d8d9/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:52:58.654772 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db417b11-6ec0-4f45-a87c-8999584d1faa admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dacdd3e7b3240aabacdf5c385d5b9b2, user_id=785d6ff244c841df97f5b3cbfc56d8d9, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:52:58.658617 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db417b11-6ec0-4f45-a87c-8999584d1faa admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:52:58.727069 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1224/2418] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:52:58 2022] PUT /identity/v3/projects/3dacdd3e7b3240aabacdf5c385d5b9b2/users/785d6ff244c841df97f5b3cbfc56d8d9/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:52:58.738481 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4d20b-a44e-4568-a73b-bf445136ed28 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:52:58.738901 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4d20b-a44e-4568-a73b-bf445136ed28 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:52:58.739097 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4d20b-a44e-4568-a73b-bf445136ed28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:52:58.771217 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-dea4d20b-a44e-4568-a73b-bf445136ed28 None None] MFA Rules not processed for user `785d6ff244c841df97f5b3cbfc56d8d9`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:52:58.773222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-dea4d20b-a44e-4568-a73b-bf445136ed28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:52:58.850026 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1195/2419] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:52:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:52:58.860747 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db417b11-6ec0-4f45-a87c-8999584d1faa admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:52:58.862490 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-db417b11-6ec0-4f45-a87c-8999584d1faa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:52:58.886958 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:52:58.925075 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db417b11-6ec0-4f45-a87c-8999584d1faa admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:52:58.928762 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4918d2-d0e5-4802-840d-f3901f911a7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:52:58.929350 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4918d2-d0e5-4802-840d-f3901f911a7b service cinder] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:52:58.929697 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4918d2-d0e5-4802-840d-f3901f911a7b service cinder] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:52:58.929894 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4918d2-d0e5-4802-840d-f3901f911a7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:52:58.941963 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4918d2-d0e5-4802-840d-f3901f911a7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:52:58.943864 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4918d2-d0e5-4802-840d-f3901f911a7b service cinder] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:52:58.956476 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1225/2420] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:52:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:53:00.773085 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb5cac-0ce7-4d1f-964e-6a3b2ce9124f None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:53:00.773578 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb5cac-0ce7-4d1f-964e-6a3b2ce9124f None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:53:00.773673 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb5cac-0ce7-4d1f-964e-6a3b2ce9124f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:53:00.794326 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-e5fb5cac-0ce7-4d1f-964e-6a3b2ce9124f None None] MFA Rules not processed for user `785d6ff244c841df97f5b3cbfc56d8d9`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:53:00.795455 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-e5fb5cac-0ce7-4d1f-964e-6a3b2ce9124f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:53:00.802625 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1196/2421] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:53:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:53:09.038568 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=17 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:53:09.693321 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814b1cf3-8a20-4698-9d9a-b9dbbf5df4d6 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:53:09.698030 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814b1cf3-8a20-4698-9d9a-b9dbbf5df4d6 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:53:09.698030 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814b1cf3-8a20-4698-9d9a-b9dbbf5df4d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:53:09.714228 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-814b1cf3-8a20-4698-9d9a-b9dbbf5df4d6 None None] MFA Rules not processed for user `bcd83a222eb04368a409db06389f1b7d`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:53:09.716054 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-814b1cf3-8a20-4698-9d9a-b9dbbf5df4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:53:09.785501 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1226/2422] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:53:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:53:10.789212 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:53:16.163675 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fb5cac-0ce7-4d1f-964e-6a3b2ce9124f None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:53:16.165294 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-e5fb5cac-0ce7-4d1f-964e-6a3b2ce9124f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:53:16.222747 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fb5cac-0ce7-4d1f-964e-6a3b2ce9124f None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:53:16.226330 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b81007f-a15f-4503-9212-aabfa63e00ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:53:16.226934 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b81007f-a15f-4503-9212-aabfa63e00ef service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:53:16.227218 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b81007f-a15f-4503-9212-aabfa63e00ef service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:53:16.227494 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b81007f-a15f-4503-9212-aabfa63e00ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:53:16.229337 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-4b81007f-a15f-4503-9212-aabfa63e00ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:53:16.299696 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b81007f-a15f-4503-9212-aabfa63e00ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:53:16.301182 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b81007f-a15f-4503-9212-aabfa63e00ef service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:53:16.312592 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1197/2423] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:53:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:53:16.439757 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814b1cf3-8a20-4698-9d9a-b9dbbf5df4d6 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:53:16.441199 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-814b1cf3-8a20-4698-9d9a-b9dbbf5df4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:53:16.454066 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814b1cf3-8a20-4698-9d9a-b9dbbf5df4d6 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:53:16.459180 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a95d09e-5d9d-4b5b-893f-3f863e065b4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:53:16.459180 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a95d09e-5d9d-4b5b-893f-3f863e065b4b service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:53:16.459180 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a95d09e-5d9d-4b5b-893f-3f863e065b4b service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:53:16.459180 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a95d09e-5d9d-4b5b-893f-3f863e065b4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:53:16.461061 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-1a95d09e-5d9d-4b5b-893f-3f863e065b4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:53:16.474897 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a95d09e-5d9d-4b5b-893f-3f863e065b4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:53:16.476360 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a95d09e-5d9d-4b5b-893f-3f863e065b4b service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:53:16.487587 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1227/2424] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:53:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:53:26.307105 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=54 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:53:26.482817 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=31 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:53:26.945601 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b81007f-a15f-4503-9212-aabfa63e00ef service neutron] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:53:26.947047 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-4b81007f-a15f-4503-9212-aabfa63e00ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:53:26.961316 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b81007f-a15f-4503-9212-aabfa63e00ef service neutron] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:53:26.967574 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05bafea-a30e-4d70-8f8b-38dcc8fc388e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:53:26.968174 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05bafea-a30e-4d70-8f8b-38dcc8fc388e service nova] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:53:26.968487 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05bafea-a30e-4d70-8f8b-38dcc8fc388e service nova] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:53:26.968861 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05bafea-a30e-4d70-8f8b-38dcc8fc388e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:53:26.970613 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-b05bafea-a30e-4d70-8f8b-38dcc8fc388e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:53:26.981132 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05bafea-a30e-4d70-8f8b-38dcc8fc388e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:53:26.983516 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05bafea-a30e-4d70-8f8b-38dcc8fc388e service nova] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:53:26.992937 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1198/2425] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:53:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:53:33.800847 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a95d09e-5d9d-4b5b-893f-3f863e065b4b service neutron] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:53:33.802287 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-1a95d09e-5d9d-4b5b-893f-3f863e065b4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:53:33.817911 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a95d09e-5d9d-4b5b-893f-3f863e065b4b service neutron] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:53:33.821733 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f731c3-c49e-4f80-9804-b785d1b64085 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:53:33.822349 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f731c3-c49e-4f80-9804-b785d1b64085 service cinder] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:53:33.822632 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f731c3-c49e-4f80-9804-b785d1b64085 service cinder] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:53:33.822908 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f731c3-c49e-4f80-9804-b785d1b64085 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:53:33.824815 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-72f731c3-c49e-4f80-9804-b785d1b64085 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:53:33.894197 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f731c3-c49e-4f80-9804-b785d1b64085 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:53:33.895629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f731c3-c49e-4f80-9804-b785d1b64085 service cinder] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:53:33.905152 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1228/2426] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:53:33 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:53:36.087957 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d75b2-955d-4eba-875c-569b795a7bc5 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:53:36.087957 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d75b2-955d-4eba-875c-569b795a7bc5 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:53:36.087957 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d75b2-955d-4eba-875c-569b795a7bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:53:36.091061 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-201d75b2-955d-4eba-875c-569b795a7bc5 None None] MFA Rules not processed for user `f77dfe78b1a8405084c06d9acee96511`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:53:36.092546 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-201d75b2-955d-4eba-875c-569b795a7bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:53:36.176887 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1199/2427] 10.222.0.52 () {58 vars in 1009 bytes} [Mon Aug 29 22:53:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:53:36.349170 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f731c3-c49e-4f80-9804-b785d1b64085 service cinder] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:53:36.350651 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-72f731c3-c49e-4f80-9804-b785d1b64085 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:53:36.423898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f731c3-c49e-4f80-9804-b785d1b64085 service cinder] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:53:36.429134 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a82ef-f3b6-4912-a7d6-94bb8920bbb9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57c175ac90e545358746176972ff39a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:53:36.429742 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a82ef-f3b6-4912-a7d6-94bb8920bbb9 service swift] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:53:36.430025 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a82ef-f3b6-4912-a7d6-94bb8920bbb9 service swift] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:53:36.430298 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a82ef-f3b6-4912-a7d6-94bb8920bbb9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:53:36.440311 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84a82ef-f3b6-4912-a7d6-94bb8920bbb9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:53:36.441921 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84a82ef-f3b6-4912-a7d6-94bb8920bbb9 service swift] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:53:36.450723 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1229/2428] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Aug 29 22:53:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 22:53:37.449529 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3deff-dac0-45bb-b89a-02c3d3ffa852 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:53:37.449969 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3deff-dac0-45bb-b89a-02c3d3ffa852 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:53:37.450112 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3deff-dac0-45bb-b89a-02c3d3ffa852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:53:37.474287 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-f8f3deff-dac0-45bb-b89a-02c3d3ffa852 None None] MFA Rules not processed for user `f77dfe78b1a8405084c06d9acee96511`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:53:37.475480 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-f8f3deff-dac0-45bb-b89a-02c3d3ffa852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:53:37.489103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1200/2429] 10.222.0.52 () {58 vars in 967 bytes} [Mon Aug 29 22:53:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:53:37.501278 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a82ef-f3b6-4912-a7d6-94bb8920bbb9 service swift] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:53:37.512193 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a82ef-f3b6-4912-a7d6-94bb8920bbb9 service swift] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:53:37.523341 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeeb795-5283-403c-9ef8-1ec4a99d7f3e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57c175ac90e545358746176972ff39a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:53:37.525965 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeeb795-5283-403c-9ef8-1ec4a99d7f3e service swift] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:53:37.525965 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeeb795-5283-403c-9ef8-1ec4a99d7f3e service swift] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:53:37.525965 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeeb795-5283-403c-9ef8-1ec4a99d7f3e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:53:37.534635 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeeb795-5283-403c-9ef8-1ec4a99d7f3e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:53:37.536253 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeeb795-5283-403c-9ef8-1ec4a99d7f3e service swift] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:53:37.545396 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1230/2430] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Aug 29 22:53:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 22:53:38.431284 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b07be1-26c7-4d11-92fc-a31bb595a01c None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:53:38.431563 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b07be1-26c7-4d11-92fc-a31bb595a01c None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:53:38.432022 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b07be1-26c7-4d11-92fc-a31bb595a01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:53:38.460244 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-43b07be1-26c7-4d11-92fc-a31bb595a01c None None] MFA Rules not processed for user `f77dfe78b1a8405084c06d9acee96511`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:53:38.464779 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-43b07be1-26c7-4d11-92fc-a31bb595a01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:53:38.473067 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1201/2431] 10.222.0.52 () {58 vars in 1009 bytes} [Mon Aug 29 22:53:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:53:38.485022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeeb795-5283-403c-9ef8-1ec4a99d7f3e service swift] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:53:38.495825 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeeb795-5283-403c-9ef8-1ec4a99d7f3e service swift] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:53:38.501123 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9795f41-3188-42d8-9533-5f02a45c2f54 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57c175ac90e545358746176972ff39a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:53:38.501727 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9795f41-3188-42d8-9533-5f02a45c2f54 service swift] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:53:38.502290 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9795f41-3188-42d8-9533-5f02a45c2f54 service swift] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:53:38.502394 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9795f41-3188-42d8-9533-5f02a45c2f54 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:53:38.512858 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9795f41-3188-42d8-9533-5f02a45c2f54 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:53:38.512858 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9795f41-3188-42d8-9533-5f02a45c2f54 service swift] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:53:38.520892 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1231/2432] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Aug 29 22:53:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 22:53:39.111763 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1108ee-2124-4495-a60a-bc9cde3543f5 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:53:39.112050 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1108ee-2124-4495-a60a-bc9cde3543f5 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:53:39.112326 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1108ee-2124-4495-a60a-bc9cde3543f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:53:39.131585 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-be1108ee-2124-4495-a60a-bc9cde3543f5 None None] MFA Rules not processed for user `f77dfe78b1a8405084c06d9acee96511`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:53:39.132799 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-be1108ee-2124-4495-a60a-bc9cde3543f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:53:39.139854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1202/2433] 10.222.0.52 () {58 vars in 967 bytes} [Mon Aug 29 22:53:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:53:39.152893 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9795f41-3188-42d8-9533-5f02a45c2f54 service swift] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:53:39.163978 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9795f41-3188-42d8-9533-5f02a45c2f54 service swift] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:53:39.169032 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f5c3a0-d115-47b6-b5de-d01d47f0a674 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57c175ac90e545358746176972ff39a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:53:39.169624 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5c3a0-d115-47b6-b5de-d01d47f0a674 service swift] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:53:39.169907 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5c3a0-d115-47b6-b5de-d01d47f0a674 service swift] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:53:39.170181 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5c3a0-d115-47b6-b5de-d01d47f0a674 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:53:39.178354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f5c3a0-d115-47b6-b5de-d01d47f0a674 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:53:39.179942 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f5c3a0-d115-47b6-b5de-d01d47f0a674 service swift] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:53:39.187916 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1232/2434] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Aug 29 22:53:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 22:53:42.001181 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1108ee-2124-4495-a60a-bc9cde3543f5 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:53:42.069922 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7bf577-f95c-4104-a87d-740fdd6defd4 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:53:42.070920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7bf577-f95c-4104-a87d-740fdd6defd4 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:53:42.071417 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7bf577-f95c-4104-a87d-740fdd6defd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:53:42.100340 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-9a7bf577-f95c-4104-a87d-740fdd6defd4 None None] MFA Rules not processed for user `70cf6f471ff04dab9f8cbc268397928e`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:53:42.101691 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-9a7bf577-f95c-4104-a87d-740fdd6defd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:53:42.117762 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1108ee-2124-4495-a60a-bc9cde3543f5 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:53:42.123699 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c14dd0-d467-4591-97b7-fcb4c3b1a04a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:53:42.124404 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c14dd0-d467-4591-97b7-fcb4c3b1a04a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:53:42.125046 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c14dd0-d467-4591-97b7-fcb4c3b1a04a admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:53:42.125324 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c14dd0-d467-4591-97b7-fcb4c3b1a04a admin admin] PATH_INFO: `/v3/users/785d6ff244c841df97f5b3cbfc56d8d9` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:53:42.126720 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c14dd0-d467-4591-97b7-fcb4c3b1a04a admin admin] RBAC: Authorizing `identity:delete_user(user_id=785d6ff244c841df97f5b3cbfc56d8d9)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:53:42.129600 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c14dd0-d467-4591-97b7-fcb4c3b1a04a admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:53:42.150840 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:53:42.164913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1233/2435] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:53:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:53:42.378642 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-25c14dd0-d467-4591-97b7-fcb4c3b1a04a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '785d6ff244c841df97f5b3cbfc56d8d9'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:53:42.418969 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7bf577-f95c-4104-a87d-740fdd6defd4 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:53:42.422886 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-25c14dd0-d467-4591-97b7-fcb4c3b1a04a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '785d6ff244c841df97f5b3cbfc56d8d9'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:53:42.430949 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7bf577-f95c-4104-a87d-740fdd6defd4 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:53:42.433355 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-25c14dd0-d467-4591-97b7-fcb4c3b1a04a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '785d6ff244c841df97f5b3cbfc56d8d9'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:53:42.434742 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264b3c88-b88a-4657-840c-28b4b641b69f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:53:42.435510 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:53:42.435662 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b3c88-b88a-4657-840c-28b4b641b69f service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:53:42.435846 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b3c88-b88a-4657-840c-28b4b641b69f service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:53:42.436267 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b3c88-b88a-4657-840c-28b4b641b69f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:53:42.438575 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-264b3c88-b88a-4657-840c-28b4b641b69f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:53:42.449601 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264b3c88-b88a-4657-840c-28b4b641b69f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:53:42.455947 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264b3c88-b88a-4657-840c-28b4b641b69f service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:53:42.459503 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1234/2436] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:53:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:53:42.513130 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-25c14dd0-d467-4591-97b7-fcb4c3b1a04a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '785d6ff244c841df97f5b3cbfc56d8d9'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:53:42.514068 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-25c14dd0-d467-4591-97b7-fcb4c3b1a04a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '785d6ff244c841df97f5b3cbfc56d8d9'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:53:42.521543 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1203/2437] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:53:42 2022] DELETE /identity/v3/users/785d6ff244c841df97f5b3cbfc56d8d9 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:53:42.526353 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9390e-722a-44ef-8b86-3603e0916753 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:53:42.526650 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9390e-722a-44ef-8b86-3603e0916753 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:53:42.526923 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9390e-722a-44ef-8b86-3603e0916753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:53:42.547164 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-c9f9390e-722a-44ef-8b86-3603e0916753 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:53:42.548425 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-c9f9390e-722a-44ef-8b86-3603e0916753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:53:42.611766 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1235/2438] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:53:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:53:42.621487 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c14dd0-d467-4591-97b7-fcb4c3b1a04a admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:53:42.622928 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-25c14dd0-d467-4591-97b7-fcb4c3b1a04a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:53:42.685841 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c14dd0-d467-4591-97b7-fcb4c3b1a04a admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:53:42.689487 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f00899-b431-4829-bf96-7bb1a9b454a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:53:42.690182 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f00899-b431-4829-bf96-7bb1a9b454a3 service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:53:42.690577 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f00899-b431-4829-bf96-7bb1a9b454a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:53:42.690706 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f00899-b431-4829-bf96-7bb1a9b454a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:53:42.701283 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f00899-b431-4829-bf96-7bb1a9b454a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:53:42.702733 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f00899-b431-4829-bf96-7bb1a9b454a3 service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:53:42.717340 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1204/2439] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:53:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:53:42.986077 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9390e-722a-44ef-8b86-3603e0916753 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:53:42.989039 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-c9f9390e-722a-44ef-8b86-3603e0916753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:53:43.061624 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f00899-b431-4829-bf96-7bb1a9b454a3 service neutron] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:53:43.063995 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-79f00899-b431-4829-bf96-7bb1a9b454a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:53:43.088169 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f00899-b431-4829-bf96-7bb1a9b454a3 service neutron] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:53:43.108625 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431da44d-02d4-456e-9467-f620b3c47839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:53:43.109317 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431da44d-02d4-456e-9467-f620b3c47839 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:53:43.109594 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431da44d-02d4-456e-9467-f620b3c47839 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:53:43.109860 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431da44d-02d4-456e-9467-f620b3c47839 admin admin] PATH_INFO: `/v3/projects/3dacdd3e7b3240aabacdf5c385d5b9b2` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:53:43.111429 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431da44d-02d4-456e-9467-f620b3c47839 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dacdd3e7b3240aabacdf5c385d5b9b2)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:53:43.116890 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431da44d-02d4-456e-9467-f620b3c47839 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:53:43.128055 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9390e-722a-44ef-8b86-3603e0916753 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:53:43.144228 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07db4b-d047-4e4c-91cd-8d60fa299526 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8fd0f6e5b144d2ca78b8d96722502a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:53:43.144898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07db4b-d047-4e4c-91cd-8d60fa299526 service placement] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:53:43.145175 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07db4b-d047-4e4c-91cd-8d60fa299526 service placement] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:53:43.145440 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07db4b-d047-4e4c-91cd-8d60fa299526 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:53:43.149512 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-ff07db4b-d047-4e4c-91cd-8d60fa299526 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:53:43.181263 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff07db4b-d047-4e4c-91cd-8d60fa299526 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:53:43.183462 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff07db4b-d047-4e4c-91cd-8d60fa299526 service placement] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:53:43.201757 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1236/2440] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:53:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:53:43.321702 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-431da44d-02d4-456e-9467-f620b3c47839 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dacdd3e7b3240aabacdf5c385d5b9b2', 'request_id': 'req-431da44d-02d4-456e-9467-f620b3c47839'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:53:43.405194 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-431da44d-02d4-456e-9467-f620b3c47839 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dacdd3e7b3240aabacdf5c385d5b9b2', 'request_id': 'req-431da44d-02d4-456e-9467-f620b3c47839'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:53:43.415143 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-431da44d-02d4-456e-9467-f620b3c47839 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dacdd3e7b3240aabacdf5c385d5b9b2', 'request_id': 'req-431da44d-02d4-456e-9467-f620b3c47839'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:53:43.433517 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-431da44d-02d4-456e-9467-f620b3c47839 admin admin] The token cache is being invalidate because project 3dacdd3e7b3240aabacdf5c385d5b9b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:53:43.433810 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-431da44d-02d4-456e-9467-f620b3c47839 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:53:43.435722 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-431da44d-02d4-456e-9467-f620b3c47839 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:53:43.442258 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1205/2441] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:53:43 2022] DELETE /identity/v3/projects/3dacdd3e7b3240aabacdf5c385d5b9b2 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:53:43.448524 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671e97c-fead-48d7-956b-b15f5f73dbad None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:53:43.452056 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671e97c-fead-48d7-956b-b15f5f73dbad None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:53:43.452056 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671e97c-fead-48d7-956b-b15f5f73dbad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:53:43.470326 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-c671e97c-fead-48d7-956b-b15f5f73dbad None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:53:43.471544 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-c671e97c-fead-48d7-956b-b15f5f73dbad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:53:43.556767 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1237/2442] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:53:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:53:43.628293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc8f93c-8fde-40ba-9c8c-c5c35f7b75f9 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:53:43.628639 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc8f93c-8fde-40ba-9c8c-c5c35f7b75f9 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:53:43.629309 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc8f93c-8fde-40ba-9c8c-c5c35f7b75f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:53:43.654685 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-bbc8f93c-8fde-40ba-9c8c-c5c35f7b75f9 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:53:43.658279 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-bbc8f93c-8fde-40ba-9c8c-c5c35f7b75f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:53:43.668564 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1206/2443] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:53:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:53:43.672738 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c671e97c-fead-48d7-956b-b15f5f73dbad None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:53:43.685229 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c671e97c-fead-48d7-956b-b15f5f73dbad None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:53:43.694036 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7168922-08b5-4f2f-9043-5f24dab0f3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:53:43.694661 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7168922-08b5-4f2f-9043-5f24dab0f3c4 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:53:43.694976 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7168922-08b5-4f2f-9043-5f24dab0f3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:53:43.695341 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7168922-08b5-4f2f-9043-5f24dab0f3c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:53:43.696035 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7168922-08b5-4f2f-9043-5f24dab0f3c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:53:43.696342 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7168922-08b5-4f2f-9043-5f24dab0f3c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:53:43.698303 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7168922-08b5-4f2f-9043-5f24dab0f3c4 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:53:43.708012 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:53:43.717542 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1238/2444] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:53:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:53:43.718777 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc8f93c-8fde-40ba-9c8c-c5c35f7b75f9 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:53:43.730463 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc8f93c-8fde-40ba-9c8c-c5c35f7b75f9 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:53:43.735880 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c5af8c-9921-4be6-9c17-a1a460820a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:53:43.736653 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c5af8c-9921-4be6-9c17-a1a460820a95 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:53:43.737026 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c5af8c-9921-4be6-9c17-a1a460820a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:53:43.737532 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c5af8c-9921-4be6-9c17-a1a460820a95 admin admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:53:43.738542 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c5af8c-9921-4be6-9c17-a1a460820a95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:53:43.740278 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c5af8c-9921-4be6-9c17-a1a460820a95 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:53:43.854765 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1207/2445] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:53:43 2022] POST /identity/v3/projects => generated 371 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:53:43.858976 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7168922-08b5-4f2f-9043-5f24dab0f3c4 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:53:43.871077 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7168922-08b5-4f2f-9043-5f24dab0f3c4 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:53:43.937398 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c5ad3-6d29-41b6-b714-e4116a9b2e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:53:43.937983 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c5ad3-6d29-41b6-b714-e4116a9b2e08 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:53:43.938264 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c5ad3-6d29-41b6-b714-e4116a9b2e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:53:43.938539 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c5ad3-6d29-41b6-b714-e4116a9b2e08 admin admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:53:43.939177 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0c5ad3-6d29-41b6-b714-e4116a9b2e08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:53:43.941969 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0c5ad3-6d29-41b6-b714-e4116a9b2e08 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:53:44.054186 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1239/2446] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:53:43 2022] POST /identity/v3/users => generated 286 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:53:44.058457 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c5af8c-9921-4be6-9c17-a1a460820a95 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:53:44.069361 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c5af8c-9921-4be6-9c17-a1a460820a95 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:53:44.074178 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7861d7e-0c70-4223-a17e-0a856e6f9ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:53:44.074790 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7861d7e-0c70-4223-a17e-0a856e6f9ffa admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:53:44.075236 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7861d7e-0c70-4223-a17e-0a856e6f9ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:53:44.075489 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7861d7e-0c70-4223-a17e-0a856e6f9ffa admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:53:44.076150 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7861d7e-0c70-4223-a17e-0a856e6f9ffa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:53:44.077810 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7861d7e-0c70-4223-a17e-0a856e6f9ffa admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:53:44.096953 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1208/2447] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:53:44 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:53:44.098374 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c5ad3-6d29-41b6-b714-e4116a9b2e08 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:53:44.109065 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c5ad3-6d29-41b6-b714-e4116a9b2e08 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:53:44.114490 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b87d57-12e9-4c16-85b2-71cc02c3c7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:53:44.115087 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b87d57-12e9-4c16-85b2-71cc02c3c7d9 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:53:44.115367 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b87d57-12e9-4c16-85b2-71cc02c3c7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:53:44.115702 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b87d57-12e9-4c16-85b2-71cc02c3c7d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:53:44.116355 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b87d57-12e9-4c16-85b2-71cc02c3c7d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:53:44.117974 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b87d57-12e9-4c16-85b2-71cc02c3c7d9 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:53:44.130040 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1240/2448] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:53:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:53:44.134392 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7861d7e-0c70-4223-a17e-0a856e6f9ffa admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:53:44.145655 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7861d7e-0c70-4223-a17e-0a856e6f9ffa admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:53:44.151791 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0525ed37-e307-4c78-b835-dfe38f38506f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:53:44.152612 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0525ed37-e307-4c78-b835-dfe38f38506f admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:53:44.152987 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0525ed37-e307-4c78-b835-dfe38f38506f admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:53:44.159540 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0525ed37-e307-4c78-b835-dfe38f38506f admin admin] PATH_INFO: `/v3/projects/e1e766828cca43808ba55f00a50c53f9/users/5f04bf8fdc2c4451a3cb6a9929273973/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:53:44.174722 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0525ed37-e307-4c78-b835-dfe38f38506f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1e766828cca43808ba55f00a50c53f9, user_id=5f04bf8fdc2c4451a3cb6a9929273973, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:53:44.177849 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0525ed37-e307-4c78-b835-dfe38f38506f admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:53:44.280316 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1209/2449] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:53:44 2022] PUT /identity/v3/projects/e1e766828cca43808ba55f00a50c53f9/users/5f04bf8fdc2c4451a3cb6a9929273973/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:53:44.285672 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d95b156-2ca8-4cd7-a962-65cf05499130 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:53:44.285984 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d95b156-2ca8-4cd7-a962-65cf05499130 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:53:44.286233 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d95b156-2ca8-4cd7-a962-65cf05499130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:53:44.306154 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-0d95b156-2ca8-4cd7-a962-65cf05499130 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:53:44.307384 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-0d95b156-2ca8-4cd7-a962-65cf05499130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:53:44.374028 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1241/2450] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:53:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:53:44.385900 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0525ed37-e307-4c78-b835-dfe38f38506f admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:53:44.389396 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-0525ed37-e307-4c78-b835-dfe38f38506f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:53:44.412280 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:53:44.494594 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0525ed37-e307-4c78-b835-dfe38f38506f admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:53:44.498617 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a4093f-0cfa-4ec7-b72c-557abe8d0a34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:53:44.499310 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4093f-0cfa-4ec7-b72c-557abe8d0a34 service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:53:44.499781 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4093f-0cfa-4ec7-b72c-557abe8d0a34 service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:53:44.500108 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4093f-0cfa-4ec7-b72c-557abe8d0a34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:53:44.511006 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a4093f-0cfa-4ec7-b72c-557abe8d0a34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:53:44.520675 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a4093f-0cfa-4ec7-b72c-557abe8d0a34 service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:53:44.522960 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1210/2451] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:53:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:53:47.653085 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d95b156-2ca8-4cd7-a962-65cf05499130 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:53:47.664252 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d95b156-2ca8-4cd7-a962-65cf05499130 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:53:47.668641 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f6b9e2-d43c-4e59-bb53-c6fc420f04ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:53:47.669554 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f6b9e2-d43c-4e59-bb53-c6fc420f04ab service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:53:47.669987 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f6b9e2-d43c-4e59-bb53-c6fc420f04ab service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:53:47.670603 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f6b9e2-d43c-4e59-bb53-c6fc420f04ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:53:47.673333 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-f8f6b9e2-d43c-4e59-bb53-c6fc420f04ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:53:47.699925 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f6b9e2-d43c-4e59-bb53-c6fc420f04ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:53:47.701429 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f6b9e2-d43c-4e59-bb53-c6fc420f04ab service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:53:47.711822 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1242/2452] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:53:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:53:51.228380 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c04a7-a81d-4867-9fec-8a1a86db292d None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:53:51.228906 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c04a7-a81d-4867-9fec-8a1a86db292d None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:53:51.228993 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c04a7-a81d-4867-9fec-8a1a86db292d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:53:51.276919 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-8d3c04a7-a81d-4867-9fec-8a1a86db292d None None] MFA Rules not processed for user `5f04bf8fdc2c4451a3cb6a9929273973`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:53:51.278427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-8d3c04a7-a81d-4867-9fec-8a1a86db292d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:53:51.374467 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1211/2453] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:53:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:53:51.379618 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f6b9e2-d43c-4e59-bb53-c6fc420f04ab service neutron] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:53:51.391054 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f6b9e2-d43c-4e59-bb53-c6fc420f04ab service neutron] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:53:51.397020 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13748b89-f97a-4e48-881f-2c993679cf74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:53:51.398204 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13748b89-f97a-4e48-881f-2c993679cf74 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:53:51.398628 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13748b89-f97a-4e48-881f-2c993679cf74 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:53:51.399017 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13748b89-f97a-4e48-881f-2c993679cf74 admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:53:51.399770 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13748b89-f97a-4e48-881f-2c993679cf74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:53:51.401884 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13748b89-f97a-4e48-881f-2c993679cf74 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:53:51.504386 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1243/2454] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:53:51 2022] POST /identity/v3/projects => generated 371 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:53:51.509192 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3c04a7-a81d-4867-9fec-8a1a86db292d None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:53:51.526718 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3c04a7-a81d-4867-9fec-8a1a86db292d None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:53:51.602325 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76116aac-111f-42d0-9114-e237acc8853e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:53:51.602451 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76116aac-111f-42d0-9114-e237acc8853e admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:53:51.602726 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76116aac-111f-42d0-9114-e237acc8853e admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:53:51.603003 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76116aac-111f-42d0-9114-e237acc8853e admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:53:51.603642 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76116aac-111f-42d0-9114-e237acc8853e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:53:51.605609 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76116aac-111f-42d0-9114-e237acc8853e admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:53:51.806009 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1212/2455] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:53:51 2022] POST /identity/v3/users => generated 292 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:53:51.809987 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13748b89-f97a-4e48-881f-2c993679cf74 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:53:51.820405 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13748b89-f97a-4e48-881f-2c993679cf74 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:53:51.825118 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a2e01-2d23-4985-9b28-0c51a5ba6591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:53:51.825763 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a2e01-2d23-4985-9b28-0c51a5ba6591 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:53:51.826110 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a2e01-2d23-4985-9b28-0c51a5ba6591 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:53:51.826389 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a2e01-2d23-4985-9b28-0c51a5ba6591 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:53:51.827069 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22a2e01-2d23-4985-9b28-0c51a5ba6591 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:53:51.828824 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22a2e01-2d23-4985-9b28-0c51a5ba6591 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:53:51.840446 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1244/2456] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:53:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:53:51.844938 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76116aac-111f-42d0-9114-e237acc8853e admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:53:51.856500 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76116aac-111f-42d0-9114-e237acc8853e admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:53:51.861196 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739d5838-bfbd-45bb-85e7-52f1d2eee503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:53:51.861856 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739d5838-bfbd-45bb-85e7-52f1d2eee503 admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:53:51.862138 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739d5838-bfbd-45bb-85e7-52f1d2eee503 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:53:51.862414 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739d5838-bfbd-45bb-85e7-52f1d2eee503 admin admin] PATH_INFO: `/v3/projects/4c5041b479ac4dcfa6d5c53793fa9156/users/f04e671081ac4c9db7c389a88fed9b42/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:53:51.878774 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739d5838-bfbd-45bb-85e7-52f1d2eee503 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c5041b479ac4dcfa6d5c53793fa9156, user_id=f04e671081ac4c9db7c389a88fed9b42, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:53:51.881485 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739d5838-bfbd-45bb-85e7-52f1d2eee503 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:53:51.966256 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1213/2457] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:53:51 2022] PUT /identity/v3/projects/4c5041b479ac4dcfa6d5c53793fa9156/users/f04e671081ac4c9db7c389a88fed9b42/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:53:51.971058 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a2e01-2d23-4985-9b28-0c51a5ba6591 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:53:51.982387 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a2e01-2d23-4985-9b28-0c51a5ba6591 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:53:52.012535 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:53:52.042676 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5c4017-ca62-498b-929a-ccaa8f26356f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:53:52.043281 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c4017-ca62-498b-929a-ccaa8f26356f admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:53:52.043738 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c4017-ca62-498b-929a-ccaa8f26356f admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:53:52.044214 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c4017-ca62-498b-929a-ccaa8f26356f admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:53:52.045000 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5c4017-ca62-498b-929a-ccaa8f26356f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:53:52.046947 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5c4017-ca62-498b-929a-ccaa8f26356f admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:53:52.061447 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1245/2458] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:53:51 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:53:52.062646 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739d5838-bfbd-45bb-85e7-52f1d2eee503 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:53:52.076206 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739d5838-bfbd-45bb-85e7-52f1d2eee503 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:53:52.082031 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb82f591-1aed-4ffb-bef7-6fff7222d085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:53:52.082697 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb82f591-1aed-4ffb-bef7-6fff7222d085 admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:53:52.082976 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb82f591-1aed-4ffb-bef7-6fff7222d085 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:53:52.083249 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb82f591-1aed-4ffb-bef7-6fff7222d085 admin admin] PATH_INFO: `/v3/projects/4c5041b479ac4dcfa6d5c53793fa9156/users/f04e671081ac4c9db7c389a88fed9b42/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:53:52.085210 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb82f591-1aed-4ffb-bef7-6fff7222d085 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c5041b479ac4dcfa6d5c53793fa9156, user_id=f04e671081ac4c9db7c389a88fed9b42, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:53:52.088265 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb82f591-1aed-4ffb-bef7-6fff7222d085 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:53:52.094112 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:53:52.100071 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1214/2459] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:53:52 2022] PUT /identity/v3/projects/4c5041b479ac4dcfa6d5c53793fa9156/users/f04e671081ac4c9db7c389a88fed9b42/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:53:54.058658 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5c4017-ca62-498b-929a-ccaa8f26356f admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:53:54.060760 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-3a5c4017-ca62-498b-929a-ccaa8f26356f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:53:54.227208 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb82f591-1aed-4ffb-bef7-6fff7222d085 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:53:54.229648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-fb82f591-1aed-4ffb-bef7-6fff7222d085 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:53:54.276644 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5c4017-ca62-498b-929a-ccaa8f26356f admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:53:54.385588 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb82f591-1aed-4ffb-bef7-6fff7222d085 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:53:54.397338 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4353601-a0f7-42dc-9b75-b8a4eb5a2ffb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:53:54.397973 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4353601-a0f7-42dc-9b75-b8a4eb5a2ffb service nova] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:53:54.398172 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4353601-a0f7-42dc-9b75-b8a4eb5a2ffb service nova] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:53:54.398459 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4353601-a0f7-42dc-9b75-b8a4eb5a2ffb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:53:54.403913 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-c4353601-a0f7-42dc-9b75-b8a4eb5a2ffb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:53:54.412970 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea75911-9b9d-4373-bd26-66f788ef906d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:53:54.413608 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea75911-9b9d-4373-bd26-66f788ef906d service nova] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:53:54.413891 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea75911-9b9d-4373-bd26-66f788ef906d service nova] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:53:54.414168 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea75911-9b9d-4373-bd26-66f788ef906d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:53:54.420587 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-9ea75911-9b9d-4373-bd26-66f788ef906d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:53:54.433020 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4353601-a0f7-42dc-9b75-b8a4eb5a2ffb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:53:54.435939 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4353601-a0f7-42dc-9b75-b8a4eb5a2ffb service nova] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:53:54.477210 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1215/2460] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:53:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:53:54.570284 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea75911-9b9d-4373-bd26-66f788ef906d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:53:54.572185 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea75911-9b9d-4373-bd26-66f788ef906d service nova] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:53:54.600591 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1246/2461] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:53:54 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:53:59.588580 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03560ec2-494e-4d0d-959a-6d9f50e33818 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:53:59.589178 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03560ec2-494e-4d0d-959a-6d9f50e33818 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:53:59.589277 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03560ec2-494e-4d0d-959a-6d9f50e33818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:53:59.637604 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-03560ec2-494e-4d0d-959a-6d9f50e33818 None None] MFA Rules not processed for user `f04e671081ac4c9db7c389a88fed9b42`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:53:59.638916 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-03560ec2-494e-4d0d-959a-6d9f50e33818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:53:59.735185 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1216/2462] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:53:59 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:53:59.750437 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea75911-9b9d-4373-bd26-66f788ef906d service nova] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:53:59.756550 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:53:59.762703 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea75911-9b9d-4373-bd26-66f788ef906d service nova] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:53:59.771504 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc5a3b-b1f1-4559-b9fd-796c4ebf3591 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:53:59.772515 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc5a3b-b1f1-4559-b9fd-796c4ebf3591 service nova] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:53:59.772974 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc5a3b-b1f1-4559-b9fd-796c4ebf3591 service nova] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:53:59.773408 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc5a3b-b1f1-4559-b9fd-796c4ebf3591 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:53:59.841234 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc5a3b-b1f1-4559-b9fd-796c4ebf3591 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:53:59.842957 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc5a3b-b1f1-4559-b9fd-796c4ebf3591 service nova] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:53:59.853234 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1247/2463] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:53:59 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:54:03.275352 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43333b2-e268-4b56-b3de-5b078f457c24 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:03.276242 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43333b2-e268-4b56-b3de-5b078f457c24 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:03.277625 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43333b2-e268-4b56-b3de-5b078f457c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:03.298583 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-d43333b2-e268-4b56-b3de-5b078f457c24 None None] MFA Rules not processed for user `5f04bf8fdc2c4451a3cb6a9929273973`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:54:03.299901 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-d43333b2-e268-4b56-b3de-5b078f457c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:03.310750 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1217/2464] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:54:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:54:03.470764 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc5a3b-b1f1-4559-b9fd-796c4ebf3591 service nova] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:03.472919 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-e9fc5a3b-b1f1-4559-b9fd-796c4ebf3591 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:03.544321 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc5a3b-b1f1-4559-b9fd-796c4ebf3591 service nova] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:03.548820 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bada934-dfed-44d0-a0d5-2d1a05095560 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:03.549566 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bada934-dfed-44d0-a0d5-2d1a05095560 service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:03.549927 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bada934-dfed-44d0-a0d5-2d1a05095560 service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:03.553100 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bada934-dfed-44d0-a0d5-2d1a05095560 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:03.555365 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-5bada934-dfed-44d0-a0d5-2d1a05095560 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:03.643993 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bada934-dfed-44d0-a0d5-2d1a05095560 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:03.645748 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bada934-dfed-44d0-a0d5-2d1a05095560 service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:03.660005 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1248/2465] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:54:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:54:07.746786 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43333b2-e268-4b56-b3de-5b078f457c24 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:07.746786 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-d43333b2-e268-4b56-b3de-5b078f457c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:07.762054 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43333b2-e268-4b56-b3de-5b078f457c24 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:07.766719 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eab354a-5332-45eb-9185-c68888aa9233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:07.767345 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab354a-5332-45eb-9185-c68888aa9233 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:07.767626 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab354a-5332-45eb-9185-c68888aa9233 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:07.767896 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab354a-5332-45eb-9185-c68888aa9233 admin admin] PATH_INFO: `/v3/users/53f9f8eb7a20431ca0d679af45946bce` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:07.769755 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eab354a-5332-45eb-9185-c68888aa9233 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53f9f8eb7a20431ca0d679af45946bce)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:07.771689 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eab354a-5332-45eb-9185-c68888aa9233 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:08.021222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-1eab354a-5332-45eb-9185-c68888aa9233 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53f9f8eb7a20431ca0d679af45946bce'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:08.029084 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:54:08.034379 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-1eab354a-5332-45eb-9185-c68888aa9233 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53f9f8eb7a20431ca0d679af45946bce'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:08.041353 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-1eab354a-5332-45eb-9185-c68888aa9233 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53f9f8eb7a20431ca0d679af45946bce'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:08.136898 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-1eab354a-5332-45eb-9185-c68888aa9233 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53f9f8eb7a20431ca0d679af45946bce'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:08.137673 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-1eab354a-5332-45eb-9185-c68888aa9233 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53f9f8eb7a20431ca0d679af45946bce'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:08.145009 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1218/2466] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:54:07 2022] DELETE /identity/v3/users/53f9f8eb7a20431ca0d679af45946bce => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:54:08.464706 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bada934-dfed-44d0-a0d5-2d1a05095560 service neutron] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:08.475434 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bada934-dfed-44d0-a0d5-2d1a05095560 service neutron] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:08.551678 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:54:08.563930 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d495279-a45d-4b13-8493-b8461376e5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:08.564568 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d495279-a45d-4b13-8493-b8461376e5df admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:08.564882 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d495279-a45d-4b13-8493-b8461376e5df admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:08.565161 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d495279-a45d-4b13-8493-b8461376e5df admin admin] PATH_INFO: `/v3/projects/52aad64da33c4a48aa9eb7e325c5669b` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:08.567278 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d495279-a45d-4b13-8493-b8461376e5df admin admin] RBAC: Authorizing `identity:delete_project(project_id=52aad64da33c4a48aa9eb7e325c5669b)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:08.569272 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d495279-a45d-4b13-8493-b8461376e5df admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:08.690160 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-6d495279-a45d-4b13-8493-b8461376e5df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52aad64da33c4a48aa9eb7e325c5669b', 'request_id': 'req-6d495279-a45d-4b13-8493-b8461376e5df'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:08.695688 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-6d495279-a45d-4b13-8493-b8461376e5df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52aad64da33c4a48aa9eb7e325c5669b', 'request_id': 'req-6d495279-a45d-4b13-8493-b8461376e5df'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:08.701960 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-6d495279-a45d-4b13-8493-b8461376e5df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52aad64da33c4a48aa9eb7e325c5669b', 'request_id': 'req-6d495279-a45d-4b13-8493-b8461376e5df'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:08.771398 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-6d495279-a45d-4b13-8493-b8461376e5df admin admin] The token cache is being invalidate because project 52aad64da33c4a48aa9eb7e325c5669b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:54:08.771617 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-6d495279-a45d-4b13-8493-b8461376e5df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:08.772250 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-6d495279-a45d-4b13-8493-b8461376e5df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:08.773851 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1249/2467] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:54:08 2022] DELETE /identity/v3/projects/52aad64da33c4a48aa9eb7e325c5669b => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:54:08.778810 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402beae7-7aed-4d30-81ee-d64df3e9af14 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:08.779300 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402beae7-7aed-4d30-81ee-d64df3e9af14 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:08.779404 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402beae7-7aed-4d30-81ee-d64df3e9af14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:08.801274 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-402beae7-7aed-4d30-81ee-d64df3e9af14 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:54:08.802485 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-402beae7-7aed-4d30-81ee-d64df3e9af14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:08.866387 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1219/2468] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:54:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:54:08.874305 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d031fb-6fa9-4383-82e3-d4957dbc0f8a None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:08.874570 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d031fb-6fa9-4383-82e3-d4957dbc0f8a None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:08.874840 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d031fb-6fa9-4383-82e3-d4957dbc0f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:08.895982 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-c1d031fb-6fa9-4383-82e3-d4957dbc0f8a None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:54:08.896898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-c1d031fb-6fa9-4383-82e3-d4957dbc0f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:08.910826 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1250/2469] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:54:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:54:08.915144 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402beae7-7aed-4d30-81ee-d64df3e9af14 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:08.928335 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402beae7-7aed-4d30-81ee-d64df3e9af14 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:08.933230 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a68d53-8884-43e0-bdf0-469fa2035cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:08.933835 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a68d53-8884-43e0-bdf0-469fa2035cda admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:08.934110 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a68d53-8884-43e0-bdf0-469fa2035cda admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:08.934384 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a68d53-8884-43e0-bdf0-469fa2035cda admin admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:08.935029 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a68d53-8884-43e0-bdf0-469fa2035cda admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:54:08.935326 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a68d53-8884-43e0-bdf0-469fa2035cda admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:08.936960 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a68d53-8884-43e0-bdf0-469fa2035cda admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:08.951202 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1220/2470] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:54:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:54:08.955644 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d031fb-6fa9-4383-82e3-d4957dbc0f8a None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:08.965830 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d031fb-6fa9-4383-82e3-d4957dbc0f8a None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:08.970531 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80500484-038e-4f69-948b-4aa6778592fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:08.971130 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80500484-038e-4f69-948b-4aa6778592fa admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:08.971408 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80500484-038e-4f69-948b-4aa6778592fa admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:08.971678 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80500484-038e-4f69-948b-4aa6778592fa admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:08.972474 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80500484-038e-4f69-948b-4aa6778592fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:08.974401 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80500484-038e-4f69-948b-4aa6778592fa admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:09.064674 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1251/2471] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:54:08 2022] POST /identity/v3/projects => generated 387 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:54:09.069774 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a68d53-8884-43e0-bdf0-469fa2035cda admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:09.080401 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a68d53-8884-43e0-bdf0-469fa2035cda admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:09.146853 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e4ba9e-df08-4ec7-a74a-abe3838057dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:09.147474 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4ba9e-df08-4ec7-a74a-abe3838057dc admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:09.147752 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4ba9e-df08-4ec7-a74a-abe3838057dc admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:09.148022 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4ba9e-df08-4ec7-a74a-abe3838057dc admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:09.148667 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e4ba9e-df08-4ec7-a74a-abe3838057dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:09.150803 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e4ba9e-df08-4ec7-a74a-abe3838057dc admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:09.239632 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1221/2472] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:54:09 2022] POST /identity/v3/users => generated 294 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:54:09.244121 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80500484-038e-4f69-948b-4aa6778592fa admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:09.256503 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80500484-038e-4f69-948b-4aa6778592fa admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:09.261507 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7fd0b-6da5-4395-8d1a-0d73ab11d53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:09.262111 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7fd0b-6da5-4395-8d1a-0d73ab11d53d admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:09.262391 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7fd0b-6da5-4395-8d1a-0d73ab11d53d admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:09.262665 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7fd0b-6da5-4395-8d1a-0d73ab11d53d admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:09.263310 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a7fd0b-6da5-4395-8d1a-0d73ab11d53d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:09.265189 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a7fd0b-6da5-4395-8d1a-0d73ab11d53d admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:09.275485 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1252/2473] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:54:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:54:09.279650 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e4ba9e-df08-4ec7-a74a-abe3838057dc admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:09.291158 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e4ba9e-df08-4ec7-a74a-abe3838057dc admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:09.296280 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93969064-ca3b-45d5-b4ce-55b6da69bf4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:09.296941 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93969064-ca3b-45d5-b4ce-55b6da69bf4a admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:09.297222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93969064-ca3b-45d5-b4ce-55b6da69bf4a admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:09.297498 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93969064-ca3b-45d5-b4ce-55b6da69bf4a admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:09.298141 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93969064-ca3b-45d5-b4ce-55b6da69bf4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:09.299743 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93969064-ca3b-45d5-b4ce-55b6da69bf4a admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:09.315898 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1222/2474] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:54:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:54:09.320736 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7fd0b-6da5-4395-8d1a-0d73ab11d53d admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:09.330585 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7fd0b-6da5-4395-8d1a-0d73ab11d53d admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:09.335250 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1e0772-7b55-4ca2-96a5-91fdc57d7515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:09.335880 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e0772-7b55-4ca2-96a5-91fdc57d7515 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:09.336157 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e0772-7b55-4ca2-96a5-91fdc57d7515 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:09.336430 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e0772-7b55-4ca2-96a5-91fdc57d7515 admin admin] PATH_INFO: `/v3/projects/b7ddc57eb893458aba3e58be198d9fac/users/90dceaf89941425dba0fcc037d27d1e7/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:09.351465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1e0772-7b55-4ca2-96a5-91fdc57d7515 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7ddc57eb893458aba3e58be198d9fac, user_id=90dceaf89941425dba0fcc037d27d1e7, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:09.354341 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1e0772-7b55-4ca2-96a5-91fdc57d7515 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:09.428893 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1253/2475] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:54:09 2022] PUT /identity/v3/projects/b7ddc57eb893458aba3e58be198d9fac/users/90dceaf89941425dba0fcc037d27d1e7/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:54:09.443573 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825866d-c94d-4af4-833b-ba56e152e8b4 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:09.443573 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825866d-c94d-4af4-833b-ba56e152e8b4 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:09.443573 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825866d-c94d-4af4-833b-ba56e152e8b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:09.474523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-c825866d-c94d-4af4-833b-ba56e152e8b4 None None] MFA Rules not processed for user `90dceaf89941425dba0fcc037d27d1e7`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:54:09.475804 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-c825866d-c94d-4af4-833b-ba56e152e8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:09.512278 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:54:09.568170 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1223/2476] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:54:09 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:54:09.576928 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1e0772-7b55-4ca2-96a5-91fdc57d7515 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:09.655775 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1e0772-7b55-4ca2-96a5-91fdc57d7515 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:09.661455 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef311bd0-33aa-43d8-99a6-5ae3d6a79334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:09.662236 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef311bd0-33aa-43d8-99a6-5ae3d6a79334 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:09.662910 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef311bd0-33aa-43d8-99a6-5ae3d6a79334 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:09.662910 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef311bd0-33aa-43d8-99a6-5ae3d6a79334 admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:09.663461 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef311bd0-33aa-43d8-99a6-5ae3d6a79334 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:09.665472 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef311bd0-33aa-43d8-99a6-5ae3d6a79334 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:09.784176 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1254/2477] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:54:09 2022] POST /identity/v3/projects => generated 387 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:54:09.789650 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825866d-c94d-4af4-833b-ba56e152e8b4 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:09.801724 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825866d-c94d-4af4-833b-ba56e152e8b4 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:09.866926 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8ebf2a-5479-48b1-b392-c4f7c911d5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:09.867538 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8ebf2a-5479-48b1-b392-c4f7c911d5c0 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:09.867846 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8ebf2a-5479-48b1-b392-c4f7c911d5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:09.868120 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8ebf2a-5479-48b1-b392-c4f7c911d5c0 admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:09.868814 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8ebf2a-5479-48b1-b392-c4f7c911d5c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:09.870737 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8ebf2a-5479-48b1-b392-c4f7c911d5c0 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:09.968783 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1224/2478] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:54:09 2022] POST /identity/v3/users => generated 300 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:54:09.973489 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef311bd0-33aa-43d8-99a6-5ae3d6a79334 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:09.979178 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:54:09.984813 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef311bd0-33aa-43d8-99a6-5ae3d6a79334 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:09.989759 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984de541-3235-4db6-9082-f8b4784ab02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:09.990384 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984de541-3235-4db6-9082-f8b4784ab02b admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:09.990739 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984de541-3235-4db6-9082-f8b4784ab02b admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:09.991015 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984de541-3235-4db6-9082-f8b4784ab02b admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:09.991698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984de541-3235-4db6-9082-f8b4784ab02b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:09.993357 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984de541-3235-4db6-9082-f8b4784ab02b admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:10.006452 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1255/2479] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:54:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:54:10.013722 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8ebf2a-5479-48b1-b392-c4f7c911d5c0 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:10.024960 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8ebf2a-5479-48b1-b392-c4f7c911d5c0 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:10.029966 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d359f4e-a704-41e2-bc47-2ed5c8daf718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:10.030803 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d359f4e-a704-41e2-bc47-2ed5c8daf718 admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:10.030999 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d359f4e-a704-41e2-bc47-2ed5c8daf718 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:10.031277 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d359f4e-a704-41e2-bc47-2ed5c8daf718 admin admin] PATH_INFO: `/v3/projects/060282785dbf477299edbf9298dbda13/users/11516ada148b483ea5d672d4b1d95a3e/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:10.047238 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d359f4e-a704-41e2-bc47-2ed5c8daf718 admin admin] RBAC: Authorizing `identity:create_grant(project_id=060282785dbf477299edbf9298dbda13, user_id=11516ada148b483ea5d672d4b1d95a3e, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:10.049947 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d359f4e-a704-41e2-bc47-2ed5c8daf718 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:10.131224 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1225/2480] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:54:10 2022] PUT /identity/v3/projects/060282785dbf477299edbf9298dbda13/users/11516ada148b483ea5d672d4b1d95a3e/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:54:10.135152 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984de541-3235-4db6-9082-f8b4784ab02b admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:10.146330 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984de541-3235-4db6-9082-f8b4784ab02b admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:10.210736 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5818e7-c42b-4ed2-be73-f22a04a3fa08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:10.211347 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5818e7-c42b-4ed2-be73-f22a04a3fa08 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:10.211880 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5818e7-c42b-4ed2-be73-f22a04a3fa08 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:10.212152 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5818e7-c42b-4ed2-be73-f22a04a3fa08 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:10.212826 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5818e7-c42b-4ed2-be73-f22a04a3fa08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:10.214432 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5818e7-c42b-4ed2-be73-f22a04a3fa08 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:10.225095 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1256/2481] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:54:10 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:54:10.229426 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d359f4e-a704-41e2-bc47-2ed5c8daf718 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:10.240468 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d359f4e-a704-41e2-bc47-2ed5c8daf718 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:10.245189 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1450e91-93e6-4816-b61c-c54a7f951ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:10.246976 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1450e91-93e6-4816-b61c-c54a7f951ab3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:10.246976 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1450e91-93e6-4816-b61c-c54a7f951ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:10.246976 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1450e91-93e6-4816-b61c-c54a7f951ab3 admin admin] PATH_INFO: `/v3/projects/060282785dbf477299edbf9298dbda13/users/11516ada148b483ea5d672d4b1d95a3e/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:10.248324 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1450e91-93e6-4816-b61c-c54a7f951ab3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=060282785dbf477299edbf9298dbda13, user_id=11516ada148b483ea5d672d4b1d95a3e, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:10.251047 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1450e91-93e6-4816-b61c-c54a7f951ab3 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:10.263458 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1226/2482] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:54:10 2022] PUT /identity/v3/projects/060282785dbf477299edbf9298dbda13/users/11516ada148b483ea5d672d4b1d95a3e/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:54:10.274727 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad381aa-d691-4ee5-979f-feb516f126e0 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:10.275011 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad381aa-d691-4ee5-979f-feb516f126e0 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:10.281503 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad381aa-d691-4ee5-979f-feb516f126e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:10.306141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-4ad381aa-d691-4ee5-979f-feb516f126e0 None None] MFA Rules not processed for user `11516ada148b483ea5d672d4b1d95a3e`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:54:10.307408 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-4ad381aa-d691-4ee5-979f-feb516f126e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:10.381487 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1257/2483] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:54:10 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:54:10.392919 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1450e91-93e6-4816-b61c-c54a7f951ab3 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:10.395816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-d1450e91-93e6-4816-b61c-c54a7f951ab3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:10.454859 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1450e91-93e6-4816-b61c-c54a7f951ab3 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:10.459271 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e64a8-f203-4d73-88fe-40b5ccb9a738 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:10.459993 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e64a8-f203-4d73-88fe-40b5ccb9a738 service cinder] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:10.460280 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e64a8-f203-4d73-88fe-40b5ccb9a738 service cinder] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:10.460693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e64a8-f203-4d73-88fe-40b5ccb9a738 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:10.473720 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746e64a8-f203-4d73-88fe-40b5ccb9a738 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:10.473872 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746e64a8-f203-4d73-88fe-40b5ccb9a738 service cinder] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:10.484111 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1227/2484] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:54:10 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:54:10.513946 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad381aa-d691-4ee5-979f-feb516f126e0 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:10.588469 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad381aa-d691-4ee5-979f-feb516f126e0 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:10.588469 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ca28b-3500-44f5-949f-9f0b4669fd68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:10.588469 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ca28b-3500-44f5-949f-9f0b4669fd68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:10.588469 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ca28b-3500-44f5-949f-9f0b4669fd68 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:10.589573 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ca28b-3500-44f5-949f-9f0b4669fd68 admin admin] PATH_INFO: `/v3/users/90dceaf89941425dba0fcc037d27d1e7` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:10.589573 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8ca28b-3500-44f5-949f-9f0b4669fd68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90dceaf89941425dba0fcc037d27d1e7)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:10.590898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8ca28b-3500-44f5-949f-9f0b4669fd68 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:10.610485 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:54:10.826947 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ca8ca28b-3500-44f5-949f-9f0b4669fd68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90dceaf89941425dba0fcc037d27d1e7'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:10.896568 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ca8ca28b-3500-44f5-949f-9f0b4669fd68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90dceaf89941425dba0fcc037d27d1e7'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:10.910653 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ca8ca28b-3500-44f5-949f-9f0b4669fd68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90dceaf89941425dba0fcc037d27d1e7'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:10.918587 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ca8ca28b-3500-44f5-949f-9f0b4669fd68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90dceaf89941425dba0fcc037d27d1e7'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:10.919917 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ca8ca28b-3500-44f5-949f-9f0b4669fd68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90dceaf89941425dba0fcc037d27d1e7'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:10.927997 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1258/2485] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:54:10 2022] DELETE /identity/v3/users/90dceaf89941425dba0fcc037d27d1e7 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:54:10.932692 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2933769a-21b0-4b7a-87a1-2cda2de6e47d None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:10.933214 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2933769a-21b0-4b7a-87a1-2cda2de6e47d None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:10.933316 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2933769a-21b0-4b7a-87a1-2cda2de6e47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:10.953400 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-2933769a-21b0-4b7a-87a1-2cda2de6e47d None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:54:10.954594 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-2933769a-21b0-4b7a-87a1-2cda2de6e47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:10.978930 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:54:11.025495 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1228/2486] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:54:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:54:11.035693 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ca28b-3500-44f5-949f-9f0b4669fd68 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:11.037216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-ca8ca28b-3500-44f5-949f-9f0b4669fd68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:11.096896 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ca28b-3500-44f5-949f-9f0b4669fd68 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:11.101584 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00d9270-161a-48be-bbac-0f907d299f99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:11.102334 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d9270-161a-48be-bbac-0f907d299f99 service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:11.102676 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d9270-161a-48be-bbac-0f907d299f99 service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:11.102998 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d9270-161a-48be-bbac-0f907d299f99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:11.112213 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00d9270-161a-48be-bbac-0f907d299f99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:11.113631 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00d9270-161a-48be-bbac-0f907d299f99 service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:11.122867 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1259/2487] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:54:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:54:11.544598 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2933769a-21b0-4b7a-87a1-2cda2de6e47d None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:11.546094 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-2933769a-21b0-4b7a-87a1-2cda2de6e47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:11.561177 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2933769a-21b0-4b7a-87a1-2cda2de6e47d None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:11.566613 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d4139-4b14-4076-ae1a-a44402354587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:11.569450 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d4139-4b14-4076-ae1a-a44402354587 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:11.569450 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d4139-4b14-4076-ae1a-a44402354587 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:11.569450 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d4139-4b14-4076-ae1a-a44402354587 admin admin] PATH_INFO: `/v3/projects/b7ddc57eb893458aba3e58be198d9fac` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:11.569450 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80d4139-4b14-4076-ae1a-a44402354587 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7ddc57eb893458aba3e58be198d9fac)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:11.571107 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80d4139-4b14-4076-ae1a-a44402354587 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:11.681743 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-f80d4139-4b14-4076-ae1a-a44402354587 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7ddc57eb893458aba3e58be198d9fac', 'request_id': 'req-f80d4139-4b14-4076-ae1a-a44402354587'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:11.764965 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-f80d4139-4b14-4076-ae1a-a44402354587 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7ddc57eb893458aba3e58be198d9fac', 'request_id': 'req-f80d4139-4b14-4076-ae1a-a44402354587'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:11.771918 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-f80d4139-4b14-4076-ae1a-a44402354587 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7ddc57eb893458aba3e58be198d9fac', 'request_id': 'req-f80d4139-4b14-4076-ae1a-a44402354587'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:11.783096 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-f80d4139-4b14-4076-ae1a-a44402354587 admin admin] The token cache is being invalidate because project b7ddc57eb893458aba3e58be198d9fac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:54:11.783386 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-f80d4139-4b14-4076-ae1a-a44402354587 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:11.784030 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-f80d4139-4b14-4076-ae1a-a44402354587 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:11.785818 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1229/2488] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:54:11 2022] DELETE /identity/v3/projects/b7ddc57eb893458aba3e58be198d9fac => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:54:11.790317 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00d9270-161a-48be-bbac-0f907d299f99 service neutron] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:11.801157 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00d9270-161a-48be-bbac-0f907d299f99 service neutron] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:11.858761 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c13749-04b9-4a60-8f78-ebc907aa4e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:11.859385 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c13749-04b9-4a60-8f78-ebc907aa4e7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:11.859668 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c13749-04b9-4a60-8f78-ebc907aa4e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:11.859945 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c13749-04b9-4a60-8f78-ebc907aa4e7c admin admin] PATH_INFO: `/v3/users/11516ada148b483ea5d672d4b1d95a3e` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:11.861137 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c13749-04b9-4a60-8f78-ebc907aa4e7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=11516ada148b483ea5d672d4b1d95a3e)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:11.863075 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c13749-04b9-4a60-8f78-ebc907aa4e7c admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:12.035278 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:54:12.044555 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-77c13749-04b9-4a60-8f78-ebc907aa4e7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11516ada148b483ea5d672d4b1d95a3e'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:12.155053 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-77c13749-04b9-4a60-8f78-ebc907aa4e7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11516ada148b483ea5d672d4b1d95a3e'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:12.170005 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-77c13749-04b9-4a60-8f78-ebc907aa4e7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11516ada148b483ea5d672d4b1d95a3e'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:12.178160 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-77c13749-04b9-4a60-8f78-ebc907aa4e7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11516ada148b483ea5d672d4b1d95a3e'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:12.178887 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-77c13749-04b9-4a60-8f78-ebc907aa4e7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11516ada148b483ea5d672d4b1d95a3e'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:12.185537 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1260/2489] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:54:11 2022] DELETE /identity/v3/users/11516ada148b483ea5d672d4b1d95a3e => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:54:12.613395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d4139-4b14-4076-ae1a-a44402354587 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:12.615689 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-f80d4139-4b14-4076-ae1a-a44402354587 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:12.631372 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d4139-4b14-4076-ae1a-a44402354587 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:12.699484 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481f1cb8-5d53-4607-aae0-e8f9ea60b445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:12.699947 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481f1cb8-5d53-4607-aae0-e8f9ea60b445 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:12.700334 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481f1cb8-5d53-4607-aae0-e8f9ea60b445 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:12.700794 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481f1cb8-5d53-4607-aae0-e8f9ea60b445 admin admin] PATH_INFO: `/v3/projects/060282785dbf477299edbf9298dbda13` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:12.702081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481f1cb8-5d53-4607-aae0-e8f9ea60b445 admin admin] RBAC: Authorizing `identity:delete_project(project_id=060282785dbf477299edbf9298dbda13)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:12.704067 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481f1cb8-5d53-4607-aae0-e8f9ea60b445 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:12.840448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-481f1cb8-5d53-4607-aae0-e8f9ea60b445 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '060282785dbf477299edbf9298dbda13', 'request_id': 'req-481f1cb8-5d53-4607-aae0-e8f9ea60b445'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:12.842207 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:54:12.919255 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-481f1cb8-5d53-4607-aae0-e8f9ea60b445 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '060282785dbf477299edbf9298dbda13', 'request_id': 'req-481f1cb8-5d53-4607-aae0-e8f9ea60b445'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:12.926212 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-481f1cb8-5d53-4607-aae0-e8f9ea60b445 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '060282785dbf477299edbf9298dbda13', 'request_id': 'req-481f1cb8-5d53-4607-aae0-e8f9ea60b445'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:12.941455 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-481f1cb8-5d53-4607-aae0-e8f9ea60b445 admin admin] The token cache is being invalidate because project 060282785dbf477299edbf9298dbda13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:54:12.941823 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-481f1cb8-5d53-4607-aae0-e8f9ea60b445 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:12.942491 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-481f1cb8-5d53-4607-aae0-e8f9ea60b445 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:12.943980 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1230/2490] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:54:12 2022] DELETE /identity/v3/projects/060282785dbf477299edbf9298dbda13 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:54:12.949156 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d683ef9-6405-4985-b290-66539dcdb1f4 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:12.949650 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d683ef9-6405-4985-b290-66539dcdb1f4 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:12.949736 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d683ef9-6405-4985-b290-66539dcdb1f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:12.970835 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-8d683ef9-6405-4985-b290-66539dcdb1f4 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:54:12.972111 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-8d683ef9-6405-4985-b290-66539dcdb1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:13.031872 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1261/2491] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:54:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:54:13.039863 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02aeb53-8e6d-41dd-969e-fce7a8b3c24c None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:13.040149 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02aeb53-8e6d-41dd-969e-fce7a8b3c24c None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:13.040422 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02aeb53-8e6d-41dd-969e-fce7a8b3c24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:13.061648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-a02aeb53-8e6d-41dd-969e-fce7a8b3c24c None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:54:13.062871 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-a02aeb53-8e6d-41dd-969e-fce7a8b3c24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:13.072614 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1231/2492] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:54:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:54:13.077419 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d683ef9-6405-4985-b290-66539dcdb1f4 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:13.087876 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d683ef9-6405-4985-b290-66539dcdb1f4 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:13.092890 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d747c14-2705-4b92-8593-4cb73a62ad12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:13.093507 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d747c14-2705-4b92-8593-4cb73a62ad12 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:13.093784 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d747c14-2705-4b92-8593-4cb73a62ad12 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:13.094058 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d747c14-2705-4b92-8593-4cb73a62ad12 admin admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:13.094749 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d747c14-2705-4b92-8593-4cb73a62ad12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:54:13.095053 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d747c14-2705-4b92-8593-4cb73a62ad12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:13.096695 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d747c14-2705-4b92-8593-4cb73a62ad12 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:13.110352 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1262/2493] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:54:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:54:13.115418 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02aeb53-8e6d-41dd-969e-fce7a8b3c24c None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:13.124531 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02aeb53-8e6d-41dd-969e-fce7a8b3c24c None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:13.129802 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4098bd-56e4-4af9-82cf-d93177377181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:13.130462 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4098bd-56e4-4af9-82cf-d93177377181 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:13.130784 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4098bd-56e4-4af9-82cf-d93177377181 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:13.131101 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4098bd-56e4-4af9-82cf-d93177377181 admin admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:13.131735 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4098bd-56e4-4af9-82cf-d93177377181 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:13.133981 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4098bd-56e4-4af9-82cf-d93177377181 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:13.242282 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1232/2494] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:54:13 2022] POST /identity/v3/projects => generated 379 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:54:13.246216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d747c14-2705-4b92-8593-4cb73a62ad12 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:13.257513 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d747c14-2705-4b92-8593-4cb73a62ad12 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:13.322973 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddb7c2d-9015-48f1-ac22-f9f7ec188111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:13.323758 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb7c2d-9015-48f1-ac22-f9f7ec188111 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:13.324124 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb7c2d-9015-48f1-ac22-f9f7ec188111 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:13.324320 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb7c2d-9015-48f1-ac22-f9f7ec188111 admin admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:13.325013 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddb7c2d-9015-48f1-ac22-f9f7ec188111 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:13.326970 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddb7c2d-9015-48f1-ac22-f9f7ec188111 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:13.480746 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1263/2495] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:54:13 2022] POST /identity/v3/users => generated 290 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:54:13.484608 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4098bd-56e4-4af9-82cf-d93177377181 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:13.495040 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4098bd-56e4-4af9-82cf-d93177377181 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:13.499480 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3b128-6b17-4d27-b4ff-0c2886c99ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:13.500072 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3b128-6b17-4d27-b4ff-0c2886c99ed7 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:13.500349 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3b128-6b17-4d27-b4ff-0c2886c99ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:13.500654 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3b128-6b17-4d27-b4ff-0c2886c99ed7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:13.501330 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf3b128-6b17-4d27-b4ff-0c2886c99ed7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:13.502926 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf3b128-6b17-4d27-b4ff-0c2886c99ed7 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:13.513477 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1233/2496] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:54:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:54:13.529483 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddb7c2d-9015-48f1-ac22-f9f7ec188111 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:13.529483 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddb7c2d-9015-48f1-ac22-f9f7ec188111 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:13.533561 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c018a13-0fb1-4b72-ad85-a59dd5d088e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:13.534157 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c018a13-0fb1-4b72-ad85-a59dd5d088e5 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:13.535179 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c018a13-0fb1-4b72-ad85-a59dd5d088e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:13.535467 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c018a13-0fb1-4b72-ad85-a59dd5d088e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:13.536208 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c018a13-0fb1-4b72-ad85-a59dd5d088e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:13.537963 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c018a13-0fb1-4b72-ad85-a59dd5d088e5 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:13.549342 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1264/2497] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:54:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:54:13.554084 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3b128-6b17-4d27-b4ff-0c2886c99ed7 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:13.564512 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3b128-6b17-4d27-b4ff-0c2886c99ed7 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:13.569201 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f27139-734e-435f-b5e1-f4181fb02769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:13.569853 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f27139-734e-435f-b5e1-f4181fb02769 admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:13.570134 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f27139-734e-435f-b5e1-f4181fb02769 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:13.570407 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f27139-734e-435f-b5e1-f4181fb02769 admin admin] PATH_INFO: `/v3/projects/a68dee5359754bb481b1b688ba8eb723/users/7717982ddb654eafbc58c6cf4871112e/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:13.585530 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f27139-734e-435f-b5e1-f4181fb02769 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a68dee5359754bb481b1b688ba8eb723, user_id=7717982ddb654eafbc58c6cf4871112e, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:13.588464 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f27139-734e-435f-b5e1-f4181fb02769 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:13.680302 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1234/2498] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:54:13 2022] PUT /identity/v3/projects/a68dee5359754bb481b1b688ba8eb723/users/7717982ddb654eafbc58c6cf4871112e/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:54:13.692291 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a0cac-b9f7-43e0-b019-dfd658096281 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:13.692680 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a0cac-b9f7-43e0-b019-dfd658096281 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:13.693054 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a0cac-b9f7-43e0-b019-dfd658096281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:13.724283 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-d48a0cac-b9f7-43e0-b019-dfd658096281 None None] MFA Rules not processed for user `7717982ddb654eafbc58c6cf4871112e`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:54:13.725566 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-d48a0cac-b9f7-43e0-b019-dfd658096281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:13.734364 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:54:13.798065 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1265/2499] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:54:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:54:13.803646 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f27139-734e-435f-b5e1-f4181fb02769 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:13.871406 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f27139-734e-435f-b5e1-f4181fb02769 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:13.876411 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b493e0a3-257f-47df-bf4a-0f5cf24e1092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:13.877053 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493e0a3-257f-47df-bf4a-0f5cf24e1092 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:13.877335 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493e0a3-257f-47df-bf4a-0f5cf24e1092 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:13.877610 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493e0a3-257f-47df-bf4a-0f5cf24e1092 admin admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:13.878335 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b493e0a3-257f-47df-bf4a-0f5cf24e1092 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:13.880257 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b493e0a3-257f-47df-bf4a-0f5cf24e1092 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:13.950225 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1235/2500] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:54:13 2022] POST /identity/v3/projects => generated 381 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:54:13.954563 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a0cac-b9f7-43e0-b019-dfd658096281 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:13.965727 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a0cac-b9f7-43e0-b019-dfd658096281 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:14.031847 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6c6dc2-b772-43b2-be85-a41b63530132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:14.032453 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c6dc2-b772-43b2-be85-a41b63530132 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:14.032752 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c6dc2-b772-43b2-be85-a41b63530132 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:14.036850 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c6dc2-b772-43b2-be85-a41b63530132 admin admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:14.036850 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6c6dc2-b772-43b2-be85-a41b63530132 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:14.036850 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6c6dc2-b772-43b2-be85-a41b63530132 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:14.119783 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1266/2501] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:54:13 2022] POST /identity/v3/users => generated 297 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:54:14.123874 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b493e0a3-257f-47df-bf4a-0f5cf24e1092 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:14.135066 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b493e0a3-257f-47df-bf4a-0f5cf24e1092 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:14.139860 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99857885-cc70-4540-88a6-7ef2e757f406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:14.140555 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99857885-cc70-4540-88a6-7ef2e757f406 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:14.140872 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99857885-cc70-4540-88a6-7ef2e757f406 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:14.141149 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99857885-cc70-4540-88a6-7ef2e757f406 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:14.141793 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99857885-cc70-4540-88a6-7ef2e757f406 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:14.143397 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99857885-cc70-4540-88a6-7ef2e757f406 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:14.154939 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1236/2502] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:54:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:54:14.159297 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6c6dc2-b772-43b2-be85-a41b63530132 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:14.169543 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6c6dc2-b772-43b2-be85-a41b63530132 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:14.174351 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d8f79-547d-4709-8591-f76b93934a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:14.174994 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d8f79-547d-4709-8591-f76b93934a81 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:14.175324 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d8f79-547d-4709-8591-f76b93934a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:14.175667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d8f79-547d-4709-8591-f76b93934a81 admin admin] PATH_INFO: `/v3/projects/94cf2c7a80b7402c98dfbfc06510f757/users/67277939d1744634ad1b2236201d38be/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:14.190269 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719d8f79-547d-4709-8591-f76b93934a81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94cf2c7a80b7402c98dfbfc06510f757, user_id=67277939d1744634ad1b2236201d38be, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:14.193527 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719d8f79-547d-4709-8591-f76b93934a81 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:14.297039 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1267/2503] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:54:14 2022] PUT /identity/v3/projects/94cf2c7a80b7402c98dfbfc06510f757/users/67277939d1744634ad1b2236201d38be/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:54:14.301949 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99857885-cc70-4540-88a6-7ef2e757f406 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:14.312501 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99857885-cc70-4540-88a6-7ef2e757f406 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:14.358707 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:54:14.372892 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad69b17-6569-4eba-932a-04678b91658f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:14.373518 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad69b17-6569-4eba-932a-04678b91658f admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:14.373940 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad69b17-6569-4eba-932a-04678b91658f admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:14.374346 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad69b17-6569-4eba-932a-04678b91658f admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:14.375293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad69b17-6569-4eba-932a-04678b91658f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:14.376987 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad69b17-6569-4eba-932a-04678b91658f admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:14.388657 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1237/2504] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:54:14 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:54:14.393302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d8f79-547d-4709-8591-f76b93934a81 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:14.404020 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d8f79-547d-4709-8591-f76b93934a81 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:14.408922 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc62b1-ce03-4d77-bc60-e2fb71189e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:14.409776 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc62b1-ce03-4d77-bc60-e2fb71189e82 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:14.410081 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc62b1-ce03-4d77-bc60-e2fb71189e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:14.410404 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc62b1-ce03-4d77-bc60-e2fb71189e82 admin admin] PATH_INFO: `/v3/projects/94cf2c7a80b7402c98dfbfc06510f757/users/67277939d1744634ad1b2236201d38be/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:14.412418 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbc62b1-ce03-4d77-bc60-e2fb71189e82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94cf2c7a80b7402c98dfbfc06510f757, user_id=67277939d1744634ad1b2236201d38be, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:14.415684 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbc62b1-ce03-4d77-bc60-e2fb71189e82 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:14.428464 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1268/2505] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:54:14 2022] PUT /identity/v3/projects/94cf2c7a80b7402c98dfbfc06510f757/users/67277939d1744634ad1b2236201d38be/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:54:14.439197 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9fb57-0043-4808-be48-d018241b057d None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:14.439645 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9fb57-0043-4808-be48-d018241b057d None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:14.439857 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9fb57-0043-4808-be48-d018241b057d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:14.471042 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-a4f9fb57-0043-4808-be48-d018241b057d None None] MFA Rules not processed for user `67277939d1744634ad1b2236201d38be`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:54:14.472497 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-a4f9fb57-0043-4808-be48-d018241b057d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:14.547427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1238/2506] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:54:14 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:54:14.560557 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc62b1-ce03-4d77-bc60-e2fb71189e82 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:14.562326 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-3fbc62b1-ce03-4d77-bc60-e2fb71189e82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:14.625585 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc62b1-ce03-4d77-bc60-e2fb71189e82 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:14.629552 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47a4b89-db74-4739-854a-0fbfaa51ef38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:14.629912 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a4b89-db74-4739-854a-0fbfaa51ef38 service cinder] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:14.630193 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a4b89-db74-4739-854a-0fbfaa51ef38 service cinder] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:14.630465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a4b89-db74-4739-854a-0fbfaa51ef38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:14.639901 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47a4b89-db74-4739-854a-0fbfaa51ef38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:14.641684 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47a4b89-db74-4739-854a-0fbfaa51ef38 service cinder] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:14.651994 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1269/2507] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:54:14 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:54:15.792596 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f9fb57-0043-4808-be48-d018241b057d None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:15.802626 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f9fb57-0043-4808-be48-d018241b057d None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:15.807324 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c28bd0-f519-4f8f-bd45-8837b61acbe6 tempest-VolumeQuotaClassesTest-1948613087 tempest-VolumeQuotaClassesTest-1948613087-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67277939d1744634ad1b2236201d38be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94cf2c7a80b7402c98dfbfc06510f757', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:15.807931 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c28bd0-f519-4f8f-bd45-8837b61acbe6 tempest-VolumeQuotaClassesTest-1948613087 tempest-VolumeQuotaClassesTest-1948613087-project-admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:15.808220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c28bd0-f519-4f8f-bd45-8837b61acbe6 tempest-VolumeQuotaClassesTest-1948613087 tempest-VolumeQuotaClassesTest-1948613087-project-admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:15.808501 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c28bd0-f519-4f8f-bd45-8837b61acbe6 tempest-VolumeQuotaClassesTest-1948613087 tempest-VolumeQuotaClassesTest-1948613087-project-admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:15.809216 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c28bd0-f519-4f8f-bd45-8837b61acbe6 tempest-VolumeQuotaClassesTest-1948613087 tempest-VolumeQuotaClassesTest-1948613087-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:54:15.809527 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c28bd0-f519-4f8f-bd45-8837b61acbe6 tempest-VolumeQuotaClassesTest-1948613087 tempest-VolumeQuotaClassesTest-1948613087-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:15.811124 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c28bd0-f519-4f8f-bd45-8837b61acbe6 tempest-VolumeQuotaClassesTest-1948613087 tempest-VolumeQuotaClassesTest-1948613087-project-admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:15.823454 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1239/2508] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:54:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:54:15.827671 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47a4b89-db74-4739-854a-0fbfaa51ef38 service cinder] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:15.833790 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:54:15.837973 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47a4b89-db74-4739-854a-0fbfaa51ef38 service cinder] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:15.842635 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d806828-52ec-4c28-b792-214545b08683 tempest-VolumeQuotaClassesTest-1948613087 tempest-VolumeQuotaClassesTest-1948613087-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67277939d1744634ad1b2236201d38be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94cf2c7a80b7402c98dfbfc06510f757', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:15.843228 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d806828-52ec-4c28-b792-214545b08683 tempest-VolumeQuotaClassesTest-1948613087 tempest-VolumeQuotaClassesTest-1948613087-project-admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:15.843516 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d806828-52ec-4c28-b792-214545b08683 tempest-VolumeQuotaClassesTest-1948613087 tempest-VolumeQuotaClassesTest-1948613087-project-admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:15.843789 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d806828-52ec-4c28-b792-214545b08683 tempest-VolumeQuotaClassesTest-1948613087 tempest-VolumeQuotaClassesTest-1948613087-project-admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:15.844414 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d806828-52ec-4c28-b792-214545b08683 tempest-VolumeQuotaClassesTest-1948613087 tempest-VolumeQuotaClassesTest-1948613087-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:15.846394 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d806828-52ec-4c28-b792-214545b08683 tempest-VolumeQuotaClassesTest-1948613087 tempest-VolumeQuotaClassesTest-1948613087-project-admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:15.961413 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1270/2509] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:54:15 2022] POST /identity/v3/projects => generated 353 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:54:15.965872 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c28bd0-f519-4f8f-bd45-8837b61acbe6 tempest-VolumeQuotaClassesTest-1948613087 tempest-VolumeQuotaClassesTest-1948613087-project-admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:15.976276 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c28bd0-f519-4f8f-bd45-8837b61acbe6 tempest-VolumeQuotaClassesTest-1948613087 tempest-VolumeQuotaClassesTest-1948613087-project-admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:16.044969 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ae26b-ab46-4e98-90b3-22a4ed1dd9fc tempest-VolumeQuotaClassesTest-1948613087 tempest-VolumeQuotaClassesTest-1948613087-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67277939d1744634ad1b2236201d38be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94cf2c7a80b7402c98dfbfc06510f757', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:16.045604 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ae26b-ab46-4e98-90b3-22a4ed1dd9fc tempest-VolumeQuotaClassesTest-1948613087 tempest-VolumeQuotaClassesTest-1948613087-project-admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:16.045886 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ae26b-ab46-4e98-90b3-22a4ed1dd9fc tempest-VolumeQuotaClassesTest-1948613087 tempest-VolumeQuotaClassesTest-1948613087-project-admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:16.046159 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ae26b-ab46-4e98-90b3-22a4ed1dd9fc tempest-VolumeQuotaClassesTest-1948613087 tempest-VolumeQuotaClassesTest-1948613087-project-admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:16.046800 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471ae26b-ab46-4e98-90b3-22a4ed1dd9fc tempest-VolumeQuotaClassesTest-1948613087 tempest-VolumeQuotaClassesTest-1948613087-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:54:16.047102 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471ae26b-ab46-4e98-90b3-22a4ed1dd9fc tempest-VolumeQuotaClassesTest-1948613087 tempest-VolumeQuotaClassesTest-1948613087-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:16.048810 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471ae26b-ab46-4e98-90b3-22a4ed1dd9fc tempest-VolumeQuotaClassesTest-1948613087 tempest-VolumeQuotaClassesTest-1948613087-project-admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:16.061383 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1240/2510] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:54:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:54:16.100845 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d806828-52ec-4c28-b792-214545b08683 tempest-VolumeQuotaClassesTest-1948613087 tempest-VolumeQuotaClassesTest-1948613087-project-admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:16.111938 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d806828-52ec-4c28-b792-214545b08683 tempest-VolumeQuotaClassesTest-1948613087 tempest-VolumeQuotaClassesTest-1948613087-project-admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:16.117176 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea1a50-0622-4de9-97ac-611b57742e1e tempest-VolumeQuotaClassesTest-1948613087 tempest-VolumeQuotaClassesTest-1948613087-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67277939d1744634ad1b2236201d38be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94cf2c7a80b7402c98dfbfc06510f757', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:16.117800 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea1a50-0622-4de9-97ac-611b57742e1e tempest-VolumeQuotaClassesTest-1948613087 tempest-VolumeQuotaClassesTest-1948613087-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:16.118085 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea1a50-0622-4de9-97ac-611b57742e1e tempest-VolumeQuotaClassesTest-1948613087 tempest-VolumeQuotaClassesTest-1948613087-project-admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:16.118360 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea1a50-0622-4de9-97ac-611b57742e1e tempest-VolumeQuotaClassesTest-1948613087 tempest-VolumeQuotaClassesTest-1948613087-project-admin] PATH_INFO: `/v3/projects/328f803896864bf1b3f734bed153fe6a` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:16.119668 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea1a50-0622-4de9-97ac-611b57742e1e tempest-VolumeQuotaClassesTest-1948613087 tempest-VolumeQuotaClassesTest-1948613087-project-admin] RBAC: Authorizing `identity:delete_project(project_id=328f803896864bf1b3f734bed153fe6a)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:16.121653 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea1a50-0622-4de9-97ac-611b57742e1e tempest-VolumeQuotaClassesTest-1948613087 tempest-VolumeQuotaClassesTest-1948613087-project-admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:16.298356 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-16ea1a50-0622-4de9-97ac-611b57742e1e tempest-VolumeQuotaClassesTest-1948613087 tempest-VolumeQuotaClassesTest-1948613087-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '328f803896864bf1b3f734bed153fe6a', 'request_id': 'req-16ea1a50-0622-4de9-97ac-611b57742e1e'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:16.308770 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-16ea1a50-0622-4de9-97ac-611b57742e1e tempest-VolumeQuotaClassesTest-1948613087 tempest-VolumeQuotaClassesTest-1948613087-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '328f803896864bf1b3f734bed153fe6a', 'request_id': 'req-16ea1a50-0622-4de9-97ac-611b57742e1e'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:16.315266 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-16ea1a50-0622-4de9-97ac-611b57742e1e tempest-VolumeQuotaClassesTest-1948613087 tempest-VolumeQuotaClassesTest-1948613087-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '328f803896864bf1b3f734bed153fe6a', 'request_id': 'req-16ea1a50-0622-4de9-97ac-611b57742e1e'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:16.444343 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-16ea1a50-0622-4de9-97ac-611b57742e1e tempest-VolumeQuotaClassesTest-1948613087 tempest-VolumeQuotaClassesTest-1948613087-project-admin] The token cache is being invalidate because project 328f803896864bf1b3f734bed153fe6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:54:16.444653 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-16ea1a50-0622-4de9-97ac-611b57742e1e tempest-VolumeQuotaClassesTest-1948613087 tempest-VolumeQuotaClassesTest-1948613087-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:16.445306 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-16ea1a50-0622-4de9-97ac-611b57742e1e tempest-VolumeQuotaClassesTest-1948613087 tempest-VolumeQuotaClassesTest-1948613087-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:16.446819 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1271/2511] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:54:16 2022] DELETE /identity/v3/projects/328f803896864bf1b3f734bed153fe6a => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:54:17.129715 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ae26b-ab46-4e98-90b3-22a4ed1dd9fc tempest-VolumeQuotaClassesTest-1948613087 tempest-VolumeQuotaClassesTest-1948613087-project-admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:17.135576 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-471ae26b-ab46-4e98-90b3-22a4ed1dd9fc tempest-VolumeQuotaClassesTest-1948613087 tempest-VolumeQuotaClassesTest-1948613087-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:17.184543 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=100 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:54:17.202574 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ae26b-ab46-4e98-90b3-22a4ed1dd9fc tempest-VolumeQuotaClassesTest-1948613087 tempest-VolumeQuotaClassesTest-1948613087-project-admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:17.261866 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855af728-b855-46d7-98d6-97ed27b52a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:17.262494 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855af728-b855-46d7-98d6-97ed27b52a0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:17.262772 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855af728-b855-46d7-98d6-97ed27b52a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:17.263068 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855af728-b855-46d7-98d6-97ed27b52a0f admin admin] PATH_INFO: `/v3/users/7717982ddb654eafbc58c6cf4871112e` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:17.264190 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855af728-b855-46d7-98d6-97ed27b52a0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7717982ddb654eafbc58c6cf4871112e)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:17.266177 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855af728-b855-46d7-98d6-97ed27b52a0f admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:17.536300 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-855af728-b855-46d7-98d6-97ed27b52a0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7717982ddb654eafbc58c6cf4871112e'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:17.555603 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-855af728-b855-46d7-98d6-97ed27b52a0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7717982ddb654eafbc58c6cf4871112e'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:17.563005 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-855af728-b855-46d7-98d6-97ed27b52a0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7717982ddb654eafbc58c6cf4871112e'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:17.650479 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-855af728-b855-46d7-98d6-97ed27b52a0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7717982ddb654eafbc58c6cf4871112e'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:17.651100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-855af728-b855-46d7-98d6-97ed27b52a0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7717982ddb654eafbc58c6cf4871112e'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:17.657570 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1241/2512] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:54:17 2022] DELETE /identity/v3/users/7717982ddb654eafbc58c6cf4871112e => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:54:17.663163 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99242811-f9ef-4903-a412-866bf2003d7a None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:17.663674 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99242811-f9ef-4903-a412-866bf2003d7a None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:17.663778 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99242811-f9ef-4903-a412-866bf2003d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:17.684343 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-99242811-f9ef-4903-a412-866bf2003d7a None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:54:17.685719 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-99242811-f9ef-4903-a412-866bf2003d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:17.749110 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1272/2513] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:54:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:54:17.759381 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855af728-b855-46d7-98d6-97ed27b52a0f admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:17.760842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-855af728-b855-46d7-98d6-97ed27b52a0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:17.819503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855af728-b855-46d7-98d6-97ed27b52a0f admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:17.823851 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c5596-715d-4ff4-9c9f-b5888a641b16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:17.824463 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c5596-715d-4ff4-9c9f-b5888a641b16 service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:17.824818 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c5596-715d-4ff4-9c9f-b5888a641b16 service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:17.825044 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c5596-715d-4ff4-9c9f-b5888a641b16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:17.834791 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c5596-715d-4ff4-9c9f-b5888a641b16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:17.836236 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c5596-715d-4ff4-9c9f-b5888a641b16 service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:17.846005 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1242/2514] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:54:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:54:18.321735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99242811-f9ef-4903-a412-866bf2003d7a None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:18.324806 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-99242811-f9ef-4903-a412-866bf2003d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:18.337893 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99242811-f9ef-4903-a412-866bf2003d7a None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:18.344034 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb3880-8482-46e7-aac1-ae070b683d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:18.344698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb3880-8482-46e7-aac1-ae070b683d93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:18.345041 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb3880-8482-46e7-aac1-ae070b683d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:18.345277 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb3880-8482-46e7-aac1-ae070b683d93 admin admin] PATH_INFO: `/v3/projects/a68dee5359754bb481b1b688ba8eb723` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:18.346692 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfb3880-8482-46e7-aac1-ae070b683d93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a68dee5359754bb481b1b688ba8eb723)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:18.349580 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfb3880-8482-46e7-aac1-ae070b683d93 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:18.463109 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-9bfb3880-8482-46e7-aac1-ae070b683d93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a68dee5359754bb481b1b688ba8eb723', 'request_id': 'req-9bfb3880-8482-46e7-aac1-ae070b683d93'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:18.469017 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-9bfb3880-8482-46e7-aac1-ae070b683d93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a68dee5359754bb481b1b688ba8eb723', 'request_id': 'req-9bfb3880-8482-46e7-aac1-ae070b683d93'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:18.470922 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:54:18.475562 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-9bfb3880-8482-46e7-aac1-ae070b683d93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a68dee5359754bb481b1b688ba8eb723', 'request_id': 'req-9bfb3880-8482-46e7-aac1-ae070b683d93'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:18.547431 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-9bfb3880-8482-46e7-aac1-ae070b683d93 admin admin] The token cache is being invalidate because project a68dee5359754bb481b1b688ba8eb723 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:54:18.547740 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-9bfb3880-8482-46e7-aac1-ae070b683d93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:18.549639 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-9bfb3880-8482-46e7-aac1-ae070b683d93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:18.549871 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1273/2515] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:54:18 2022] DELETE /identity/v3/projects/a68dee5359754bb481b1b688ba8eb723 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:54:18.554122 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c5596-715d-4ff4-9c9f-b5888a641b16 service neutron] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:18.564288 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c5596-715d-4ff4-9c9f-b5888a641b16 service neutron] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:18.581124 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:54:18.620901 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be061fd3-05db-45a3-981a-e377840c16f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:18.621526 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be061fd3-05db-45a3-981a-e377840c16f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:18.621806 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be061fd3-05db-45a3-981a-e377840c16f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:18.622078 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be061fd3-05db-45a3-981a-e377840c16f6 admin admin] PATH_INFO: `/v3/users/67277939d1744634ad1b2236201d38be` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:18.623250 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be061fd3-05db-45a3-981a-e377840c16f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67277939d1744634ad1b2236201d38be)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:18.625281 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be061fd3-05db-45a3-981a-e377840c16f6 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:18.841900 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-be061fd3-05db-45a3-981a-e377840c16f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67277939d1744634ad1b2236201d38be'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:18.854840 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-be061fd3-05db-45a3-981a-e377840c16f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67277939d1744634ad1b2236201d38be'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:18.862818 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-be061fd3-05db-45a3-981a-e377840c16f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67277939d1744634ad1b2236201d38be'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:18.965305 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-be061fd3-05db-45a3-981a-e377840c16f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67277939d1744634ad1b2236201d38be'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:18.965921 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-be061fd3-05db-45a3-981a-e377840c16f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67277939d1744634ad1b2236201d38be'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:18.972464 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1243/2516] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:54:18 2022] DELETE /identity/v3/users/67277939d1744634ad1b2236201d38be => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:54:19.459408 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb3880-8482-46e7-aac1-ae070b683d93 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:19.460956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-9bfb3880-8482-46e7-aac1-ae070b683d93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:19.475358 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb3880-8482-46e7-aac1-ae070b683d93 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:19.534116 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d081e463-4219-4fb9-a2a6-b0c7c1ddbe83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:19.534755 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081e463-4219-4fb9-a2a6-b0c7c1ddbe83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:19.535042 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081e463-4219-4fb9-a2a6-b0c7c1ddbe83 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:19.536007 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081e463-4219-4fb9-a2a6-b0c7c1ddbe83 admin admin] PATH_INFO: `/v3/projects/94cf2c7a80b7402c98dfbfc06510f757` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:19.537186 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d081e463-4219-4fb9-a2a6-b0c7c1ddbe83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94cf2c7a80b7402c98dfbfc06510f757)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:19.539129 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d081e463-4219-4fb9-a2a6-b0c7c1ddbe83 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:19.702049 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d081e463-4219-4fb9-a2a6-b0c7c1ddbe83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94cf2c7a80b7402c98dfbfc06510f757', 'request_id': 'req-d081e463-4219-4fb9-a2a6-b0c7c1ddbe83'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:19.708052 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d081e463-4219-4fb9-a2a6-b0c7c1ddbe83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94cf2c7a80b7402c98dfbfc06510f757', 'request_id': 'req-d081e463-4219-4fb9-a2a6-b0c7c1ddbe83'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:19.715277 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d081e463-4219-4fb9-a2a6-b0c7c1ddbe83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94cf2c7a80b7402c98dfbfc06510f757', 'request_id': 'req-d081e463-4219-4fb9-a2a6-b0c7c1ddbe83'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:19.788006 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d081e463-4219-4fb9-a2a6-b0c7c1ddbe83 admin admin] The token cache is being invalidate because project 94cf2c7a80b7402c98dfbfc06510f757 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:54:19.788290 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d081e463-4219-4fb9-a2a6-b0c7c1ddbe83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:19.790837 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d081e463-4219-4fb9-a2a6-b0c7c1ddbe83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:19.790837 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1274/2517] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:54:19 2022] DELETE /identity/v3/projects/94cf2c7a80b7402c98dfbfc06510f757 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:54:19.796385 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e699add1-e246-4cdd-b1fb-11147c015842 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:19.799786 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e699add1-e246-4cdd-b1fb-11147c015842 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:19.799786 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e699add1-e246-4cdd-b1fb-11147c015842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:19.820480 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-e699add1-e246-4cdd-b1fb-11147c015842 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:54:19.821774 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-e699add1-e246-4cdd-b1fb-11147c015842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:19.883020 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1244/2518] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:54:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:54:19.891136 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808295a5-3183-49ca-8812-a4a9d9dd402d None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:19.891423 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808295a5-3183-49ca-8812-a4a9d9dd402d None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:19.891703 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808295a5-3183-49ca-8812-a4a9d9dd402d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:19.912201 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-808295a5-3183-49ca-8812-a4a9d9dd402d None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:54:19.913451 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-808295a5-3183-49ca-8812-a4a9d9dd402d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:19.921343 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1275/2519] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:54:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:54:19.926193 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e699add1-e246-4cdd-b1fb-11147c015842 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:19.935802 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e699add1-e246-4cdd-b1fb-11147c015842 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:19.940465 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dcffaf-eec2-44e5-87dd-c2559be89685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:19.941104 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dcffaf-eec2-44e5-87dd-c2559be89685 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:19.941921 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dcffaf-eec2-44e5-87dd-c2559be89685 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:19.942268 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dcffaf-eec2-44e5-87dd-c2559be89685 admin admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:19.942931 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dcffaf-eec2-44e5-87dd-c2559be89685 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:54:19.943234 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dcffaf-eec2-44e5-87dd-c2559be89685 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:19.944876 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dcffaf-eec2-44e5-87dd-c2559be89685 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:19.957960 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1245/2520] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:54:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:54:19.963155 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808295a5-3183-49ca-8812-a4a9d9dd402d None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:19.972561 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808295a5-3183-49ca-8812-a4a9d9dd402d None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:19.977332 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ddbb4b-235a-4445-944a-158765bcee84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:19.977929 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ddbb4b-235a-4445-944a-158765bcee84 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:19.978205 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ddbb4b-235a-4445-944a-158765bcee84 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:19.978477 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ddbb4b-235a-4445-944a-158765bcee84 admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:19.979098 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ddbb4b-235a-4445-944a-158765bcee84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:19.981009 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ddbb4b-235a-4445-944a-158765bcee84 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:20.076052 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1276/2521] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:54:19 2022] POST /identity/v3/projects => generated 361 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:54:20.080472 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dcffaf-eec2-44e5-87dd-c2559be89685 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:20.091650 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dcffaf-eec2-44e5-87dd-c2559be89685 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:20.117302 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:54:20.150863 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d313e-33c2-41bb-82a4-35c73f1d288c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:20.151483 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d313e-33c2-41bb-82a4-35c73f1d288c admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:20.151763 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d313e-33c2-41bb-82a4-35c73f1d288c admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:20.152038 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d313e-33c2-41bb-82a4-35c73f1d288c admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:20.152668 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d313e-33c2-41bb-82a4-35c73f1d288c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:20.154651 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d313e-33c2-41bb-82a4-35c73f1d288c admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:20.313723 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1246/2522] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:54:20 2022] POST /identity/v3/users => generated 281 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:54:20.318674 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ddbb4b-235a-4445-944a-158765bcee84 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:20.329370 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ddbb4b-235a-4445-944a-158765bcee84 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:20.334056 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b6c65-f48f-48cd-8b1f-cffc202ab5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:20.334653 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b6c65-f48f-48cd-8b1f-cffc202ab5b4 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:20.335119 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b6c65-f48f-48cd-8b1f-cffc202ab5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:20.342321 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b6c65-f48f-48cd-8b1f-cffc202ab5b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:20.342321 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6b6c65-f48f-48cd-8b1f-cffc202ab5b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:20.342321 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6b6c65-f48f-48cd-8b1f-cffc202ab5b4 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:20.348068 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1277/2523] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:54:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:54:20.352014 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d313e-33c2-41bb-82a4-35c73f1d288c admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:20.361816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d313e-33c2-41bb-82a4-35c73f1d288c admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:20.367350 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99305e16-264a-41c7-96a8-63bd0e548cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:20.367945 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99305e16-264a-41c7-96a8-63bd0e548cc0 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:20.368229 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99305e16-264a-41c7-96a8-63bd0e548cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:20.368506 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99305e16-264a-41c7-96a8-63bd0e548cc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:20.369197 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99305e16-264a-41c7-96a8-63bd0e548cc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:20.370825 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99305e16-264a-41c7-96a8-63bd0e548cc0 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:20.382394 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1247/2524] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:54:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:54:20.386455 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b6c65-f48f-48cd-8b1f-cffc202ab5b4 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:20.396326 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b6c65-f48f-48cd-8b1f-cffc202ab5b4 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:20.401090 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec232b4-501e-4cd9-9ade-1be901aab170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:20.401749 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec232b4-501e-4cd9-9ade-1be901aab170 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:20.402031 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec232b4-501e-4cd9-9ade-1be901aab170 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:20.402386 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec232b4-501e-4cd9-9ade-1be901aab170 admin admin] PATH_INFO: `/v3/projects/f207f82b855444859128232ec8007a88/users/a96e99605435406080b63fa2a5f48d81/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:20.417645 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec232b4-501e-4cd9-9ade-1be901aab170 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f207f82b855444859128232ec8007a88, user_id=a96e99605435406080b63fa2a5f48d81, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:20.420427 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec232b4-501e-4cd9-9ade-1be901aab170 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:20.551897 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1278/2525] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:54:20 2022] PUT /identity/v3/projects/f207f82b855444859128232ec8007a88/users/a96e99605435406080b63fa2a5f48d81/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:54:20.563592 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cb528-db43-47d4-b2e3-8a6d2e83b412 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:20.563872 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cb528-db43-47d4-b2e3-8a6d2e83b412 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:20.564212 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cb528-db43-47d4-b2e3-8a6d2e83b412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:20.596280 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-a65cb528-db43-47d4-b2e3-8a6d2e83b412 None None] MFA Rules not processed for user `a96e99605435406080b63fa2a5f48d81`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:54:20.600391 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-a65cb528-db43-47d4-b2e3-8a6d2e83b412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:20.675127 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1248/2526] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:54:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:54:20.797362 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec232b4-501e-4cd9-9ade-1be901aab170 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:20.799139 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-3ec232b4-501e-4cd9-9ade-1be901aab170 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:20.818817 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:54:20.859848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec232b4-501e-4cd9-9ade-1be901aab170 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:20.863497 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731bd782-85da-4f0e-bf1b-1260f59fed22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:20.864093 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731bd782-85da-4f0e-bf1b-1260f59fed22 service cinder] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:20.864374 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731bd782-85da-4f0e-bf1b-1260f59fed22 service cinder] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:20.864648 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731bd782-85da-4f0e-bf1b-1260f59fed22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:20.872935 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731bd782-85da-4f0e-bf1b-1260f59fed22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:20.874445 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731bd782-85da-4f0e-bf1b-1260f59fed22 service cinder] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:20.884229 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1279/2527] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:54:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:54:20.900255 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65cb528-db43-47d4-b2e3-8a6d2e83b412 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:20.971761 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65cb528-db43-47d4-b2e3-8a6d2e83b412 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:20.976679 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fddb11-da61-4048-8488-d9cf7a9b077c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:20.977342 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fddb11-da61-4048-8488-d9cf7a9b077c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:20.977628 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fddb11-da61-4048-8488-d9cf7a9b077c admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:20.977915 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fddb11-da61-4048-8488-d9cf7a9b077c admin admin] PATH_INFO: `/v3/users/a96e99605435406080b63fa2a5f48d81` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:20.979057 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fddb11-da61-4048-8488-d9cf7a9b077c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a96e99605435406080b63fa2a5f48d81)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:20.981041 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fddb11-da61-4048-8488-d9cf7a9b077c admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:21.171332 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:54:21.180305 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-12fddb11-da61-4048-8488-d9cf7a9b077c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a96e99605435406080b63fa2a5f48d81'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:21.195105 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-12fddb11-da61-4048-8488-d9cf7a9b077c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a96e99605435406080b63fa2a5f48d81'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:21.204987 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-12fddb11-da61-4048-8488-d9cf7a9b077c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a96e99605435406080b63fa2a5f48d81'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:21.290720 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-12fddb11-da61-4048-8488-d9cf7a9b077c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a96e99605435406080b63fa2a5f48d81'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:21.291280 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-12fddb11-da61-4048-8488-d9cf7a9b077c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a96e99605435406080b63fa2a5f48d81'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:21.297937 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1249/2528] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:54:20 2022] DELETE /identity/v3/users/a96e99605435406080b63fa2a5f48d81 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:54:21.303265 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8edc383-5534-4a33-96b6-9505afba9b19 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:21.303640 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8edc383-5534-4a33-96b6-9505afba9b19 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:21.303997 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8edc383-5534-4a33-96b6-9505afba9b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:21.324316 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-f8edc383-5534-4a33-96b6-9505afba9b19 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:54:21.325628 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-f8edc383-5534-4a33-96b6-9505afba9b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:21.384256 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1280/2529] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:54:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:54:21.394084 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fddb11-da61-4048-8488-d9cf7a9b077c admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:21.395524 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-12fddb11-da61-4048-8488-d9cf7a9b077c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:21.451384 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fddb11-da61-4048-8488-d9cf7a9b077c admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:21.454970 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3cbc06-4a85-4795-863f-184355fa92ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:21.455556 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3cbc06-4a85-4795-863f-184355fa92ec service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:21.455831 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3cbc06-4a85-4795-863f-184355fa92ec service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:21.456098 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3cbc06-4a85-4795-863f-184355fa92ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:21.465275 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3cbc06-4a85-4795-863f-184355fa92ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:21.466681 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3cbc06-4a85-4795-863f-184355fa92ec service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:21.476488 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1250/2530] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:54:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:54:21.916112 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8edc383-5534-4a33-96b6-9505afba9b19 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:21.917620 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-f8edc383-5534-4a33-96b6-9505afba9b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:21.931831 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8edc383-5534-4a33-96b6-9505afba9b19 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:21.936763 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b11b62-6b03-48ce-8c0f-d2c5b10d9ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:21.937976 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b11b62-6b03-48ce-8c0f-d2c5b10d9ea7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:21.938256 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b11b62-6b03-48ce-8c0f-d2c5b10d9ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:21.938529 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b11b62-6b03-48ce-8c0f-d2c5b10d9ea7 admin admin] PATH_INFO: `/v3/projects/f207f82b855444859128232ec8007a88` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:21.939719 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b11b62-6b03-48ce-8c0f-d2c5b10d9ea7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f207f82b855444859128232ec8007a88)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:21.943335 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b11b62-6b03-48ce-8c0f-d2c5b10d9ea7 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:22.034945 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d4b11b62-6b03-48ce-8c0f-d2c5b10d9ea7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f207f82b855444859128232ec8007a88', 'request_id': 'req-d4b11b62-6b03-48ce-8c0f-d2c5b10d9ea7'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:22.041691 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d4b11b62-6b03-48ce-8c0f-d2c5b10d9ea7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f207f82b855444859128232ec8007a88', 'request_id': 'req-d4b11b62-6b03-48ce-8c0f-d2c5b10d9ea7'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:22.047938 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d4b11b62-6b03-48ce-8c0f-d2c5b10d9ea7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f207f82b855444859128232ec8007a88', 'request_id': 'req-d4b11b62-6b03-48ce-8c0f-d2c5b10d9ea7'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:22.159053 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d4b11b62-6b03-48ce-8c0f-d2c5b10d9ea7 admin admin] The token cache is being invalidate because project f207f82b855444859128232ec8007a88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:54:22.159347 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d4b11b62-6b03-48ce-8c0f-d2c5b10d9ea7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:22.159956 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d4b11b62-6b03-48ce-8c0f-d2c5b10d9ea7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:54:22.161518 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1281/2531] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:54:21 2022] DELETE /identity/v3/projects/f207f82b855444859128232ec8007a88 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:54:22.167261 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4be7f-5d56-4a83-8e20-d6d91aa56cdf None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:22.167725 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4be7f-5d56-4a83-8e20-d6d91aa56cdf None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:22.167821 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4be7f-5d56-4a83-8e20-d6d91aa56cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:22.187545 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-d0a4be7f-5d56-4a83-8e20-d6d91aa56cdf None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:54:22.188766 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-d0a4be7f-5d56-4a83-8e20-d6d91aa56cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:22.255245 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1251/2532] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:54:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:54:22.263011 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b8e899-751b-48da-a7fc-e7d2c3440ec6 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:22.263302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b8e899-751b-48da-a7fc-e7d2c3440ec6 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:22.263582 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b8e899-751b-48da-a7fc-e7d2c3440ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:22.286478 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-61b8e899-751b-48da-a7fc-e7d2c3440ec6 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:54:22.287304 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-61b8e899-751b-48da-a7fc-e7d2c3440ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:22.299856 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1282/2533] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:54:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:54:22.304500 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a4be7f-5d56-4a83-8e20-d6d91aa56cdf None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:22.314745 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a4be7f-5d56-4a83-8e20-d6d91aa56cdf None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:22.320252 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebd6724-b660-400a-89f0-4435bf15f427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:22.324067 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd6724-b660-400a-89f0-4435bf15f427 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:22.324067 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd6724-b660-400a-89f0-4435bf15f427 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:22.324067 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd6724-b660-400a-89f0-4435bf15f427 admin admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:22.324067 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebd6724-b660-400a-89f0-4435bf15f427 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:54:22.324067 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebd6724-b660-400a-89f0-4435bf15f427 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:22.324492 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebd6724-b660-400a-89f0-4435bf15f427 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:22.340850 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1252/2534] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:54:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:54:22.347078 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b8e899-751b-48da-a7fc-e7d2c3440ec6 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:22.352555 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b8e899-751b-48da-a7fc-e7d2c3440ec6 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:22.357347 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61068fc-f70f-40bf-90e5-9bdebd80eeb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:22.357887 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61068fc-f70f-40bf-90e5-9bdebd80eeb9 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:22.358170 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61068fc-f70f-40bf-90e5-9bdebd80eeb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:22.358446 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61068fc-f70f-40bf-90e5-9bdebd80eeb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:22.359080 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61068fc-f70f-40bf-90e5-9bdebd80eeb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:22.361032 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61068fc-f70f-40bf-90e5-9bdebd80eeb9 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:22.470412 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1283/2535] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:54:22 2022] POST /identity/v3/projects => generated 363 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:54:22.474304 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebd6724-b660-400a-89f0-4435bf15f427 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:22.484424 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebd6724-b660-400a-89f0-4435bf15f427 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:22.526252 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:54:22.542202 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bafa9b-132e-4ff9-8672-aeb756c3190e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:22.542815 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bafa9b-132e-4ff9-8672-aeb756c3190e admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:22.543143 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bafa9b-132e-4ff9-8672-aeb756c3190e admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:22.543421 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bafa9b-132e-4ff9-8672-aeb756c3190e admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:22.544213 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bafa9b-132e-4ff9-8672-aeb756c3190e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:22.546168 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bafa9b-132e-4ff9-8672-aeb756c3190e admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:22.635597 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1253/2536] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:54:22 2022] POST /identity/v3/users => generated 282 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:54:22.640151 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61068fc-f70f-40bf-90e5-9bdebd80eeb9 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:22.651028 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61068fc-f70f-40bf-90e5-9bdebd80eeb9 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:22.656379 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d592553-a80d-4f7f-bb43-c2b01cc19a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:22.657072 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d592553-a80d-4f7f-bb43-c2b01cc19a37 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:22.657359 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d592553-a80d-4f7f-bb43-c2b01cc19a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:22.657637 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d592553-a80d-4f7f-bb43-c2b01cc19a37 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:22.658314 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d592553-a80d-4f7f-bb43-c2b01cc19a37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:22.659908 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d592553-a80d-4f7f-bb43-c2b01cc19a37 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:22.670635 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1284/2537] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:54:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:54:22.674793 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bafa9b-132e-4ff9-8672-aeb756c3190e admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:22.685853 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bafa9b-132e-4ff9-8672-aeb756c3190e admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:22.690736 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3baa37-7770-4460-85c6-73691cb3f233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:22.691330 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3baa37-7770-4460-85c6-73691cb3f233 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:22.691613 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3baa37-7770-4460-85c6-73691cb3f233 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:22.691896 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3baa37-7770-4460-85c6-73691cb3f233 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:22.693329 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3baa37-7770-4460-85c6-73691cb3f233 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:22.694955 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3baa37-7770-4460-85c6-73691cb3f233 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:22.705740 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1254/2538] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:54:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:54:22.709915 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d592553-a80d-4f7f-bb43-c2b01cc19a37 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:22.715396 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:54:22.720756 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d592553-a80d-4f7f-bb43-c2b01cc19a37 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:22.725772 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a273d90c-6642-4175-af63-5398c98dfee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:22.726422 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273d90c-6642-4175-af63-5398c98dfee9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:22.726705 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273d90c-6642-4175-af63-5398c98dfee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:22.726985 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273d90c-6642-4175-af63-5398c98dfee9 admin admin] PATH_INFO: `/v3/projects/3ffc871d6a734c70ad0ad5a957f3ba24/users/9a0af717807647128b233c2da292c566/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:22.742449 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a273d90c-6642-4175-af63-5398c98dfee9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ffc871d6a734c70ad0ad5a957f3ba24, user_id=9a0af717807647128b233c2da292c566, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:22.745649 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a273d90c-6642-4175-af63-5398c98dfee9 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:22.827853 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1285/2539] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:54:22 2022] PUT /identity/v3/projects/3ffc871d6a734c70ad0ad5a957f3ba24/users/9a0af717807647128b233c2da292c566/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:54:22.839359 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dab01a-2b6b-4dad-ad0f-67ccdae503e1 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:22.839645 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dab01a-2b6b-4dad-ad0f-67ccdae503e1 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:22.840022 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dab01a-2b6b-4dad-ad0f-67ccdae503e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:22.870243 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-e5dab01a-2b6b-4dad-ad0f-67ccdae503e1 None None] MFA Rules not processed for user `9a0af717807647128b233c2da292c566`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:54:22.871503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-e5dab01a-2b6b-4dad-ad0f-67ccdae503e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:22.940397 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1255/2540] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:54:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:54:22.953029 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a273d90c-6642-4175-af63-5398c98dfee9 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:22.954744 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-a273d90c-6642-4175-af63-5398c98dfee9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:23.024363 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a273d90c-6642-4175-af63-5398c98dfee9 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:23.028619 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476610f0-0426-4fef-bfc2-f506018ca220 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:23.029249 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476610f0-0426-4fef-bfc2-f506018ca220 service cinder] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:23.029533 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476610f0-0426-4fef-bfc2-f506018ca220 service cinder] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:23.029819 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476610f0-0426-4fef-bfc2-f506018ca220 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:23.039181 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476610f0-0426-4fef-bfc2-f506018ca220 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:23.040608 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476610f0-0426-4fef-bfc2-f506018ca220 service cinder] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:23.050158 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1286/2541] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:54:22 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:54:24.767727 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dab01a-2b6b-4dad-ad0f-67ccdae503e1 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:24.769412 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-e5dab01a-2b6b-4dad-ad0f-67ccdae503e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:24.827315 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dab01a-2b6b-4dad-ad0f-67ccdae503e1 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:24.830970 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5846877b-b17f-49d1-aa8b-54c13e3ee33c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:24.831572 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846877b-b17f-49d1-aa8b-54c13e3ee33c service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:24.831851 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846877b-b17f-49d1-aa8b-54c13e3ee33c service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:24.832124 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846877b-b17f-49d1-aa8b-54c13e3ee33c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:24.833992 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-5846877b-b17f-49d1-aa8b-54c13e3ee33c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:24.843881 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5846877b-b17f-49d1-aa8b-54c13e3ee33c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:24.846072 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5846877b-b17f-49d1-aa8b-54c13e3ee33c service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:24.854543 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1256/2542] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:54:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:54:25.819391 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484973d8-f219-409b-9deb-888b6605df01 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:25.819932 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484973d8-f219-409b-9deb-888b6605df01 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:25.819932 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484973d8-f219-409b-9deb-888b6605df01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:25.848066 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-484973d8-f219-409b-9deb-888b6605df01 None None] MFA Rules not processed for user `f77dfe78b1a8405084c06d9acee96511`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:54:25.850729 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-484973d8-f219-409b-9deb-888b6605df01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:26.046812 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1287/2543] 10.222.0.52 () {58 vars in 1009 bytes} [Mon Aug 29 22:54:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:54:26.060016 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5846877b-b17f-49d1-aa8b-54c13e3ee33c service neutron] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:26.061454 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-5846877b-b17f-49d1-aa8b-54c13e3ee33c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:26.127371 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:54:26.154045 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5846877b-b17f-49d1-aa8b-54c13e3ee33c service neutron] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:26.160052 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f820db7a-be58-49c4-aea5-e85e198e6cf6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57c175ac90e545358746176972ff39a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:26.160838 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f820db7a-be58-49c4-aea5-e85e198e6cf6 service swift] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:26.161376 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f820db7a-be58-49c4-aea5-e85e198e6cf6 service swift] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:26.162530 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f820db7a-be58-49c4-aea5-e85e198e6cf6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:26.171730 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f820db7a-be58-49c4-aea5-e85e198e6cf6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:26.173456 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f820db7a-be58-49c4-aea5-e85e198e6cf6 service swift] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:26.183443 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1257/2544] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Aug 29 22:54:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 29 22:54:26.373657 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484973d8-f219-409b-9deb-888b6605df01 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:26.375813 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-484973d8-f219-409b-9deb-888b6605df01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:26.507182 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484973d8-f219-409b-9deb-888b6605df01 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:26.513839 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771932dd-f00f-43a5-8a63-ce481c1f3567 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:26.514677 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771932dd-f00f-43a5-8a63-ce481c1f3567 service nova] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:26.515524 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771932dd-f00f-43a5-8a63-ce481c1f3567 service nova] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:26.515927 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771932dd-f00f-43a5-8a63-ce481c1f3567 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:26.518146 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-771932dd-f00f-43a5-8a63-ce481c1f3567 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:26.580921 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771932dd-f00f-43a5-8a63-ce481c1f3567 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:26.589143 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771932dd-f00f-43a5-8a63-ce481c1f3567 service nova] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:26.649268 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1288/2545] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:54:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:54:36.228565 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=7 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:54:36.617186 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=94,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:54:48.683150 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713ffe8-754e-48b0-aa6c-0eb6ea7d9f91 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:48.683648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713ffe8-754e-48b0-aa6c-0eb6ea7d9f91 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:48.683759 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713ffe8-754e-48b0-aa6c-0eb6ea7d9f91 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:48.686010 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1258/2546] 10.222.0.52 () {54 vars in 921 bytes} [Mon Aug 29 22:54:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:54:48.691780 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8042a5-05f6-4ac7-aad4-7cc7ac6ab6e3 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:48.692451 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8042a5-05f6-4ac7-aad4-7cc7ac6ab6e3 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:48.692451 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8042a5-05f6-4ac7-aad4-7cc7ac6ab6e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:48.731178 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-ed8042a5-05f6-4ac7-aad4-7cc7ac6ab6e3 None None] MFA Rules not processed for user `bdbf961e02934e88af3a72d6218f32fa`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:54:48.731178 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-ed8042a5-05f6-4ac7-aad4-7cc7ac6ab6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:48.738670 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1289/2547] 10.222.0.52 () {58 vars in 1018 bytes} [Mon Aug 29 22:54:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:54:48.753641 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6713ffe8-754e-48b0-aa6c-0eb6ea7d9f91 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:48.754633 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-6713ffe8-754e-48b0-aa6c-0eb6ea7d9f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:48.836378 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6713ffe8-754e-48b0-aa6c-0eb6ea7d9f91 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:48.842809 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca52b1fd-5aeb-4d24-b646-f343aaacc3fa service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c2569e7cd774199aa5750fd6a850060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:48.843542 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca52b1fd-5aeb-4d24-b646-f343aaacc3fa service glance] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:48.843840 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca52b1fd-5aeb-4d24-b646-f343aaacc3fa service glance] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:48.844150 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca52b1fd-5aeb-4d24-b646-f343aaacc3fa service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:48.859075 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca52b1fd-5aeb-4d24-b646-f343aaacc3fa service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:48.861375 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca52b1fd-5aeb-4d24-b646-f343aaacc3fa service glance] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:48.894176 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1259/2548] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:54:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:54:49.124988 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0749924-65a4-4f4b-a68e-a722cbc02543 None None] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:49.125271 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0749924-65a4-4f4b-a68e-a722cbc02543 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:49.125737 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0749924-65a4-4f4b-a68e-a722cbc02543 None None] PATH_INFO: `/` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:49.127268 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1290/2549] 10.222.0.52 () {54 vars in 921 bytes} [Mon Aug 29 22:54:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:54:49.135107 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44819e-6e0c-44c7-9b0c-05c492d58746 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:49.140688 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44819e-6e0c-44c7-9b0c-05c492d58746 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:49.140688 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44819e-6e0c-44c7-9b0c-05c492d58746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:49.179883 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-aa44819e-6e0c-44c7-9b0c-05c492d58746 None None] MFA Rules not processed for user `0fe5e8c4cb324ffb9d5b80f18d123727`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:54:49.181605 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-aa44819e-6e0c-44c7-9b0c-05c492d58746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:49.188286 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1260/2550] 10.222.0.52 () {58 vars in 1018 bytes} [Mon Aug 29 22:54:49 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:54:49.205185 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0749924-65a4-4f4b-a68e-a722cbc02543 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:49.216420 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0749924-65a4-4f4b-a68e-a722cbc02543 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:49.220576 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c05ce-4211-4867-b85e-d95c7b815b20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:49.221671 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c05ce-4211-4867-b85e-d95c7b815b20 service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:49.221973 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c05ce-4211-4867-b85e-d95c7b815b20 service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:49.222245 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c05ce-4211-4867-b85e-d95c7b815b20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:49.230631 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90c05ce-4211-4867-b85e-d95c7b815b20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:49.232155 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90c05ce-4211-4867-b85e-d95c7b815b20 service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:49.240090 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1291/2551] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:54:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:54:49.338839 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f0fc0-82d2-4057-a9d8-10694704d954 None None] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:49.339137 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f0fc0-82d2-4057-a9d8-10694704d954 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:49.339373 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f0fc0-82d2-4057-a9d8-10694704d954 None None] PATH_INFO: `/` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:49.341144 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1261/2552] 10.222.0.52 () {54 vars in 921 bytes} [Mon Aug 29 22:54:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:54:49.351641 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ac3c0-ef8a-48a3-9595-8668aac2ab1e None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:49.352323 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ac3c0-ef8a-48a3-9595-8668aac2ab1e None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:49.352837 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ac3c0-ef8a-48a3-9595-8668aac2ab1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:49.376017 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-5e8ac3c0-ef8a-48a3-9595-8668aac2ab1e None None] MFA Rules not processed for user `bdbf961e02934e88af3a72d6218f32fa`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:54:49.383216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-5e8ac3c0-ef8a-48a3-9595-8668aac2ab1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:49.398109 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1292/2553] 10.222.0.52 () {58 vars in 1018 bytes} [Mon Aug 29 22:54:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:54:49.410827 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8f0fc0-82d2-4057-a9d8-10694704d954 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:49.420089 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-cf8f0fc0-82d2-4057-a9d8-10694704d954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:49.440178 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8f0fc0-82d2-4057-a9d8-10694704d954 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:49.447825 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ffde1-460e-4ff4-a866-e0ee04067350 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:49.448770 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ffde1-460e-4ff4-a866-e0ee04067350 service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:49.449231 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ffde1-460e-4ff4-a866-e0ee04067350 service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:49.454307 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ffde1-460e-4ff4-a866-e0ee04067350 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:49.475480 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08ffde1-460e-4ff4-a866-e0ee04067350 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:49.492294 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08ffde1-460e-4ff4-a866-e0ee04067350 service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:49.508392 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1262/2554] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:54:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:54:55.394100 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ac3c0-ef8a-48a3-9595-8668aac2ab1e None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:55.406876 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ac3c0-ef8a-48a3-9595-8668aac2ab1e None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:55.411818 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7209172a-56ad-4a90-ab1a-582cde8149ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:55.412645 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7209172a-56ad-4a90-ab1a-582cde8149ee service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:55.413049 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7209172a-56ad-4a90-ab1a-582cde8149ee service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:55.413945 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7209172a-56ad-4a90-ab1a-582cde8149ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:55.416078 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-7209172a-56ad-4a90-ab1a-582cde8149ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:55.501102 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7209172a-56ad-4a90-ab1a-582cde8149ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:55.502530 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7209172a-56ad-4a90-ab1a-582cde8149ee service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:55.513722 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1293/2555] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:54:55 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:54:57.704488 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70afdc2-2643-48a9-a106-88cf78d6f3fc None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:57.704488 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70afdc2-2643-48a9-a106-88cf78d6f3fc None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:57.704488 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70afdc2-2643-48a9-a106-88cf78d6f3fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:57.728256 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-d70afdc2-2643-48a9-a106-88cf78d6f3fc None None] MFA Rules not processed for user `f77dfe78b1a8405084c06d9acee96511`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:54:57.729852 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-d70afdc2-2643-48a9-a106-88cf78d6f3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:57.738205 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1263/2556] 10.222.0.52 () {58 vars in 1009 bytes} [Mon Aug 29 22:54:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:54:57.750949 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7209172a-56ad-4a90-ab1a-582cde8149ee service neutron] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:57.763082 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7209172a-56ad-4a90-ab1a-582cde8149ee service neutron] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:57.768304 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eafa42-cdcf-4456-8aa2-1a9844c52521 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57c175ac90e545358746176972ff39a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:57.768950 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eafa42-cdcf-4456-8aa2-1a9844c52521 service swift] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:57.769232 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eafa42-cdcf-4456-8aa2-1a9844c52521 service swift] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:57.769506 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eafa42-cdcf-4456-8aa2-1a9844c52521 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:57.778340 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eafa42-cdcf-4456-8aa2-1a9844c52521 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:57.780196 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eafa42-cdcf-4456-8aa2-1a9844c52521 service swift] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:57.788734 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1294/2557] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Aug 29 22:54:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 29 22:54:58.066507 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70afdc2-2643-48a9-a106-88cf78d6f3fc None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:58.092096 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70afdc2-2643-48a9-a106-88cf78d6f3fc None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:58.111159 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b938b4-fdc7-4e75-82d8-4a6795beb19e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:58.112336 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b938b4-fdc7-4e75-82d8-4a6795beb19e service nova] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:58.112803 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b938b4-fdc7-4e75-82d8-4a6795beb19e service nova] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:58.114080 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b938b4-fdc7-4e75-82d8-4a6795beb19e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:58.117931 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-79b938b4-fdc7-4e75-82d8-4a6795beb19e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:54:58.135481 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b938b4-fdc7-4e75-82d8-4a6795beb19e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:58.137818 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b938b4-fdc7-4e75-82d8-4a6795beb19e service nova] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:58.179032 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1264/2558] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:54:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:55:06.982534 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eafa42-cdcf-4456-8aa2-1a9844c52521 service swift] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:55:06.983999 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-98eafa42-cdcf-4456-8aa2-1a9844c52521 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:55:07.054329 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eafa42-cdcf-4456-8aa2-1a9844c52521 service swift] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:55:07.059471 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d2b8e0-7d7b-40b9-991d-89fe4feb8f4b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8fd0f6e5b144d2ca78b8d96722502a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:55:07.060066 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2b8e0-7d7b-40b9-991d-89fe4feb8f4b service placement] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:07.060343 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2b8e0-7d7b-40b9-991d-89fe4feb8f4b service placement] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:07.060612 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2b8e0-7d7b-40b9-991d-89fe4feb8f4b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:07.062352 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-05d2b8e0-7d7b-40b9-991d-89fe4feb8f4b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:55:07.074238 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d2b8e0-7d7b-40b9-991d-89fe4feb8f4b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:55:07.082920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d2b8e0-7d7b-40b9-991d-89fe4feb8f4b service placement] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:55:07.085330 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1295/2559] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:55:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:55:07.117446 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b938b4-fdc7-4e75-82d8-4a6795beb19e service nova] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:55:07.119588 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-79b938b4-fdc7-4e75-82d8-4a6795beb19e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:55:07.183907 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b938b4-fdc7-4e75-82d8-4a6795beb19e service nova] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:55:07.188510 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301c1aff-d741-40e2-80fd-292b6cd22220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:55:07.191791 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c1aff-d741-40e2-80fd-292b6cd22220 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:07.191791 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c1aff-d741-40e2-80fd-292b6cd22220 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:07.191791 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c1aff-d741-40e2-80fd-292b6cd22220 admin admin] PATH_INFO: `/v3/users/70cf6f471ff04dab9f8cbc268397928e` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:07.191791 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301c1aff-d741-40e2-80fd-292b6cd22220 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70cf6f471ff04dab9f8cbc268397928e)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:55:07.192683 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301c1aff-d741-40e2-80fd-292b6cd22220 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:55:07.372768 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-301c1aff-d741-40e2-80fd-292b6cd22220 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70cf6f471ff04dab9f8cbc268397928e'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:07.384765 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-301c1aff-d741-40e2-80fd-292b6cd22220 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70cf6f471ff04dab9f8cbc268397928e'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:07.389550 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:55:07.392097 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-301c1aff-d741-40e2-80fd-292b6cd22220 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70cf6f471ff04dab9f8cbc268397928e'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:07.433476 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-301c1aff-d741-40e2-80fd-292b6cd22220 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70cf6f471ff04dab9f8cbc268397928e'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:07.434268 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-301c1aff-d741-40e2-80fd-292b6cd22220 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70cf6f471ff04dab9f8cbc268397928e'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:07.442169 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1265/2560] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:55:07 2022] DELETE /identity/v3/users/70cf6f471ff04dab9f8cbc268397928e => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:55:07.754432 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d2b8e0-7d7b-40b9-991d-89fe4feb8f4b service placement] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:55:07.771909 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d2b8e0-7d7b-40b9-991d-89fe4feb8f4b service placement] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:55:07.821710 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=100 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:55:07.834263 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41effa64-0bc8-4c59-9494-7d8b06deef44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:55:07.834963 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41effa64-0bc8-4c59-9494-7d8b06deef44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:07.835301 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41effa64-0bc8-4c59-9494-7d8b06deef44 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:07.835584 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41effa64-0bc8-4c59-9494-7d8b06deef44 admin admin] PATH_INFO: `/v3/projects/a8c824e8723341808544a577b85d9b2d` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:07.836924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41effa64-0bc8-4c59-9494-7d8b06deef44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8c824e8723341808544a577b85d9b2d)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:55:07.840000 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41effa64-0bc8-4c59-9494-7d8b06deef44 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:55:07.949042 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-41effa64-0bc8-4c59-9494-7d8b06deef44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8c824e8723341808544a577b85d9b2d', 'request_id': 'req-41effa64-0bc8-4c59-9494-7d8b06deef44'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:07.954683 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-41effa64-0bc8-4c59-9494-7d8b06deef44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8c824e8723341808544a577b85d9b2d', 'request_id': 'req-41effa64-0bc8-4c59-9494-7d8b06deef44'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:07.960964 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-41effa64-0bc8-4c59-9494-7d8b06deef44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8c824e8723341808544a577b85d9b2d', 'request_id': 'req-41effa64-0bc8-4c59-9494-7d8b06deef44'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:08.033721 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-41effa64-0bc8-4c59-9494-7d8b06deef44 admin admin] The token cache is being invalidate because project a8c824e8723341808544a577b85d9b2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:55:08.037012 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-41effa64-0bc8-4c59-9494-7d8b06deef44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:08.037012 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-41effa64-0bc8-4c59-9494-7d8b06deef44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:08.037012 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1296/2561] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:55:07 2022] DELETE /identity/v3/projects/a8c824e8723341808544a577b85d9b2d => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:55:10.174600 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301c1aff-d741-40e2-80fd-292b6cd22220 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:55:10.176092 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-301c1aff-d741-40e2-80fd-292b6cd22220 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:55:10.190125 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301c1aff-d741-40e2-80fd-292b6cd22220 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:55:10.248850 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc65a5c2-6dea-49c0-98aa-cc7b1b707631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:55:10.249557 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc65a5c2-6dea-49c0-98aa-cc7b1b707631 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:10.249926 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc65a5c2-6dea-49c0-98aa-cc7b1b707631 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:10.250271 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc65a5c2-6dea-49c0-98aa-cc7b1b707631 admin admin] PATH_INFO: `/v3/users/9a0af717807647128b233c2da292c566` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:10.251486 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc65a5c2-6dea-49c0-98aa-cc7b1b707631 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a0af717807647128b233c2da292c566)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:55:10.253618 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc65a5c2-6dea-49c0-98aa-cc7b1b707631 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:55:10.429747 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-fc65a5c2-6dea-49c0-98aa-cc7b1b707631 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a0af717807647128b233c2da292c566'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:10.469466 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-fc65a5c2-6dea-49c0-98aa-cc7b1b707631 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a0af717807647128b233c2da292c566'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:10.502259 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-fc65a5c2-6dea-49c0-98aa-cc7b1b707631 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a0af717807647128b233c2da292c566'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:10.570971 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-fc65a5c2-6dea-49c0-98aa-cc7b1b707631 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a0af717807647128b233c2da292c566'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:10.572129 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-fc65a5c2-6dea-49c0-98aa-cc7b1b707631 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a0af717807647128b233c2da292c566'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:10.581046 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1266/2562] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:55:10 2022] DELETE /identity/v3/users/9a0af717807647128b233c2da292c566 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:55:10.586253 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03835e-0438-401e-b713-cf3f818a2430 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:10.587358 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03835e-0438-401e-b713-cf3f818a2430 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:10.588058 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03835e-0438-401e-b713-cf3f818a2430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:10.610798 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-fd03835e-0438-401e-b713-cf3f818a2430 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:55:10.612772 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-fd03835e-0438-401e-b713-cf3f818a2430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:55:10.695895 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1297/2563] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:55:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:55:10.706601 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc65a5c2-6dea-49c0-98aa-cc7b1b707631 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:55:10.708100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-fc65a5c2-6dea-49c0-98aa-cc7b1b707631 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:55:10.770455 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc65a5c2-6dea-49c0-98aa-cc7b1b707631 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:55:10.774419 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94be4f6-f8e9-4beb-ac64-50df366ee4f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:55:10.775476 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94be4f6-f8e9-4beb-ac64-50df366ee4f1 service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:10.775900 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94be4f6-f8e9-4beb-ac64-50df366ee4f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:10.776248 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94be4f6-f8e9-4beb-ac64-50df366ee4f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:10.786740 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94be4f6-f8e9-4beb-ac64-50df366ee4f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:55:10.788256 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94be4f6-f8e9-4beb-ac64-50df366ee4f1 service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:55:10.825341 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1267/2564] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:55:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:55:10.955267 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a949aa2a-1aae-4b7d-9c88-eca5c272af10 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:10.955494 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a949aa2a-1aae-4b7d-9c88-eca5c272af10 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:10.955786 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a949aa2a-1aae-4b7d-9c88-eca5c272af10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:10.975571 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-a949aa2a-1aae-4b7d-9c88-eca5c272af10 None None] MFA Rules not processed for user `f77dfe78b1a8405084c06d9acee96511`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:55:10.976987 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-a949aa2a-1aae-4b7d-9c88-eca5c272af10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:55:11.055412 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1298/2565] 10.222.0.52 () {58 vars in 967 bytes} [Mon Aug 29 22:55:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:55:11.073307 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94be4f6-f8e9-4beb-ac64-50df366ee4f1 service neutron] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:55:11.074832 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-e94be4f6-f8e9-4beb-ac64-50df366ee4f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:55:11.081937 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:55:11.169447 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94be4f6-f8e9-4beb-ac64-50df366ee4f1 service neutron] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:55:11.174806 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049db68f-93b6-40c0-92fa-b139dc5dc0c0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57c175ac90e545358746176972ff39a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:55:11.175457 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049db68f-93b6-40c0-92fa-b139dc5dc0c0 service swift] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:11.175707 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049db68f-93b6-40c0-92fa-b139dc5dc0c0 service swift] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:11.176073 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049db68f-93b6-40c0-92fa-b139dc5dc0c0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:11.185272 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049db68f-93b6-40c0-92fa-b139dc5dc0c0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:55:11.190063 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049db68f-93b6-40c0-92fa-b139dc5dc0c0 service swift] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:55:11.195919 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1268/2566] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Aug 29 22:55:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 22:55:11.281850 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a949aa2a-1aae-4b7d-9c88-eca5c272af10 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:55:11.283937 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-a949aa2a-1aae-4b7d-9c88-eca5c272af10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:55:11.299738 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a949aa2a-1aae-4b7d-9c88-eca5c272af10 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:55:11.305082 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5168561-a33c-447f-bea4-0e3cc99f8a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:55:11.305713 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5168561-a33c-447f-bea4-0e3cc99f8a0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:11.306038 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5168561-a33c-447f-bea4-0e3cc99f8a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:11.306308 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5168561-a33c-447f-bea4-0e3cc99f8a0b admin admin] PATH_INFO: `/v3/projects/3ffc871d6a734c70ad0ad5a957f3ba24` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:11.307665 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5168561-a33c-447f-bea4-0e3cc99f8a0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ffc871d6a734c70ad0ad5a957f3ba24)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:55:11.309609 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5168561-a33c-447f-bea4-0e3cc99f8a0b admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:55:11.330927 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:55:11.439234 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d5168561-a33c-447f-bea4-0e3cc99f8a0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ffc871d6a734c70ad0ad5a957f3ba24', 'request_id': 'req-d5168561-a33c-447f-bea4-0e3cc99f8a0b'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:11.446875 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d5168561-a33c-447f-bea4-0e3cc99f8a0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ffc871d6a734c70ad0ad5a957f3ba24', 'request_id': 'req-d5168561-a33c-447f-bea4-0e3cc99f8a0b'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:11.454620 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d5168561-a33c-447f-bea4-0e3cc99f8a0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ffc871d6a734c70ad0ad5a957f3ba24', 'request_id': 'req-d5168561-a33c-447f-bea4-0e3cc99f8a0b'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:11.561839 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d5168561-a33c-447f-bea4-0e3cc99f8a0b admin admin] The token cache is being invalidate because project 3ffc871d6a734c70ad0ad5a957f3ba24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:55:11.562140 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d5168561-a33c-447f-bea4-0e3cc99f8a0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:11.562931 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d5168561-a33c-447f-bea4-0e3cc99f8a0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:11.568078 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1299/2567] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:55:11 2022] DELETE /identity/v3/projects/3ffc871d6a734c70ad0ad5a957f3ba24 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:55:11.569562 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c8aaa-8457-4012-83a3-1829cb7a974e None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:11.575484 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c8aaa-8457-4012-83a3-1829cb7a974e None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:11.575909 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c8aaa-8457-4012-83a3-1829cb7a974e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:11.603202 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-306c8aaa-8457-4012-83a3-1829cb7a974e None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:55:11.604416 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-306c8aaa-8457-4012-83a3-1829cb7a974e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:55:11.695565 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1269/2568] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:55:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:55:11.704052 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49c38a-a171-4337-9dca-8c29380c9f31 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:11.704965 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49c38a-a171-4337-9dca-8c29380c9f31 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:11.705262 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49c38a-a171-4337-9dca-8c29380c9f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:11.733491 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-ca49c38a-a171-4337-9dca-8c29380c9f31 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:55:11.733491 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-ca49c38a-a171-4337-9dca-8c29380c9f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:55:11.745512 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1300/2569] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:55:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:55:11.749505 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c8aaa-8457-4012-83a3-1829cb7a974e None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:55:11.762837 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c8aaa-8457-4012-83a3-1829cb7a974e None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:55:11.797000 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189c7679-2af3-4569-986f-b9ae7b480d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:55:11.797000 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c7679-2af3-4569-986f-b9ae7b480d77 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:11.797000 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c7679-2af3-4569-986f-b9ae7b480d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:11.797000 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c7679-2af3-4569-986f-b9ae7b480d77 admin admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:11.797380 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189c7679-2af3-4569-986f-b9ae7b480d77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:55:11.797380 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189c7679-2af3-4569-986f-b9ae7b480d77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:55:11.797380 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189c7679-2af3-4569-986f-b9ae7b480d77 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:55:11.805304 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1270/2570] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:55:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:55:11.806995 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49c38a-a171-4337-9dca-8c29380c9f31 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:55:11.825287 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49c38a-a171-4337-9dca-8c29380c9f31 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:55:11.830599 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c87da-4302-49af-ad60-520c12f2bb87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:55:11.831316 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c87da-4302-49af-ad60-520c12f2bb87 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:11.831597 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c87da-4302-49af-ad60-520c12f2bb87 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:11.831868 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c87da-4302-49af-ad60-520c12f2bb87 admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:11.832506 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1c87da-4302-49af-ad60-520c12f2bb87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:55:11.834556 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1c87da-4302-49af-ad60-520c12f2bb87 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:55:11.964617 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1301/2571] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:55:11 2022] POST /identity/v3/projects => generated 377 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:55:11.966188 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189c7679-2af3-4569-986f-b9ae7b480d77 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:55:11.978444 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189c7679-2af3-4569-986f-b9ae7b480d77 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:55:12.053720 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa635115-1218-4c40-be58-fb85798e8801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:55:12.054488 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa635115-1218-4c40-be58-fb85798e8801 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:12.054840 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa635115-1218-4c40-be58-fb85798e8801 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:12.055232 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa635115-1218-4c40-be58-fb85798e8801 admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:12.056383 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa635115-1218-4c40-be58-fb85798e8801 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:55:12.058536 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa635115-1218-4c40-be58-fb85798e8801 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:55:12.144654 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1271/2572] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:55:11 2022] POST /identity/v3/users => generated 289 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:55:12.147253 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c87da-4302-49af-ad60-520c12f2bb87 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:55:12.158975 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c87da-4302-49af-ad60-520c12f2bb87 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:55:12.164219 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff03e0e7-bdbe-4bc9-9e2d-e872f9c49c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:55:12.164859 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03e0e7-bdbe-4bc9-9e2d-e872f9c49c90 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:12.165747 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03e0e7-bdbe-4bc9-9e2d-e872f9c49c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:12.166013 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03e0e7-bdbe-4bc9-9e2d-e872f9c49c90 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:12.166652 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff03e0e7-bdbe-4bc9-9e2d-e872f9c49c90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:55:12.168225 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff03e0e7-bdbe-4bc9-9e2d-e872f9c49c90 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:55:12.179408 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1302/2573] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:55:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:55:12.186978 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa635115-1218-4c40-be58-fb85798e8801 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:55:12.197330 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa635115-1218-4c40-be58-fb85798e8801 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:55:12.203523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b832ffb-59ad-41a2-b873-bd4994224ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:55:12.204242 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b832ffb-59ad-41a2-b873-bd4994224ec3 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:12.204583 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b832ffb-59ad-41a2-b873-bd4994224ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:12.204951 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b832ffb-59ad-41a2-b873-bd4994224ec3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:12.205683 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b832ffb-59ad-41a2-b873-bd4994224ec3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:55:12.207606 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b832ffb-59ad-41a2-b873-bd4994224ec3 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:55:12.214718 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:55:12.221312 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1272/2574] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:55:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:55:12.225552 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff03e0e7-bdbe-4bc9-9e2d-e872f9c49c90 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:55:12.236691 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff03e0e7-bdbe-4bc9-9e2d-e872f9c49c90 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:55:12.242233 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e456666-8b8d-48b7-bf99-a83c5ad48c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:55:12.243190 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e456666-8b8d-48b7-bf99-a83c5ad48c0f admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:12.243331 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e456666-8b8d-48b7-bf99-a83c5ad48c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:12.243567 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e456666-8b8d-48b7-bf99-a83c5ad48c0f admin admin] PATH_INFO: `/v3/projects/98941ac6692045859aa7639192603764/users/df23a24f4d5948deb641de1cefad1f69/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:12.260364 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e456666-8b8d-48b7-bf99-a83c5ad48c0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=98941ac6692045859aa7639192603764, user_id=df23a24f4d5948deb641de1cefad1f69, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:55:12.263036 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e456666-8b8d-48b7-bf99-a83c5ad48c0f admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:55:12.360613 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1303/2575] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:55:12 2022] PUT /identity/v3/projects/98941ac6692045859aa7639192603764/users/df23a24f4d5948deb641de1cefad1f69/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:55:12.366249 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c1416-2cdf-4fef-8041-56382deafccd None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:12.366623 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c1416-2cdf-4fef-8041-56382deafccd None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:12.366964 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c1416-2cdf-4fef-8041-56382deafccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:12.392234 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-626c1416-2cdf-4fef-8041-56382deafccd None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:55:12.393711 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-626c1416-2cdf-4fef-8041-56382deafccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:55:12.475527 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1273/2576] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:55:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:55:12.486409 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e456666-8b8d-48b7-bf99-a83c5ad48c0f admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:55:12.488485 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-0e456666-8b8d-48b7-bf99-a83c5ad48c0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:55:12.579345 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e456666-8b8d-48b7-bf99-a83c5ad48c0f admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:55:12.582634 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0661a0-740f-4d0e-bd8c-41c54f728c0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:55:12.583376 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0661a0-740f-4d0e-bd8c-41c54f728c0d service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:12.583752 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0661a0-740f-4d0e-bd8c-41c54f728c0d service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:12.584125 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0661a0-740f-4d0e-bd8c-41c54f728c0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:12.597924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0661a0-740f-4d0e-bd8c-41c54f728c0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:55:12.602491 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0661a0-740f-4d0e-bd8c-41c54f728c0d service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:55:12.623115 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1304/2577] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:55:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:55:14.277667 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266f7ef-95c7-4e65-b25e-1194b6240ce8 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:14.283250 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266f7ef-95c7-4e65-b25e-1194b6240ce8 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:14.283250 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266f7ef-95c7-4e65-b25e-1194b6240ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:14.312364 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-4266f7ef-95c7-4e65-b25e-1194b6240ce8 None None] MFA Rules not processed for user `f77dfe78b1a8405084c06d9acee96511`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:55:14.314708 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-4266f7ef-95c7-4e65-b25e-1194b6240ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:55:14.387842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1274/2578] 10.222.0.52 () {58 vars in 967 bytes} [Mon Aug 29 22:55:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:55:14.402915 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0661a0-740f-4d0e-bd8c-41c54f728c0d service neutron] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:55:14.404590 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-ef0661a0-740f-4d0e-bd8c-41c54f728c0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:55:14.459756 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:55:14.489413 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0661a0-740f-4d0e-bd8c-41c54f728c0d service neutron] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:55:14.495717 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce737e4f-259a-4dc0-80da-ee1a8551c836 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57c175ac90e545358746176972ff39a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:55:14.496118 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce737e4f-259a-4dc0-80da-ee1a8551c836 service swift] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:14.496525 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce737e4f-259a-4dc0-80da-ee1a8551c836 service swift] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:14.497136 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce737e4f-259a-4dc0-80da-ee1a8551c836 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:14.506381 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce737e4f-259a-4dc0-80da-ee1a8551c836 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:55:14.508105 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce737e4f-259a-4dc0-80da-ee1a8551c836 service swift] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:55:14.519168 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1305/2579] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Aug 29 22:55:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 22:55:17.251496 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4266f7ef-95c7-4e65-b25e-1194b6240ce8 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:55:17.263813 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4266f7ef-95c7-4e65-b25e-1194b6240ce8 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:55:17.268466 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a341f80-6e3c-48e1-9e01-e7fdb7975377 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:55:17.269282 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a341f80-6e3c-48e1-9e01-e7fdb7975377 service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:17.269771 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a341f80-6e3c-48e1-9e01-e7fdb7975377 service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:17.270135 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a341f80-6e3c-48e1-9e01-e7fdb7975377 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:17.272138 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-8a341f80-6e3c-48e1-9e01-e7fdb7975377 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:55:17.292415 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a341f80-6e3c-48e1-9e01-e7fdb7975377 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:55:17.294119 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a341f80-6e3c-48e1-9e01-e7fdb7975377 service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:55:17.307648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1275/2580] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:55:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:55:22.636987 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce737e4f-259a-4dc0-80da-ee1a8551c836 service swift] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:55:22.638932 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-ce737e4f-259a-4dc0-80da-ee1a8551c836 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:55:22.835551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce737e4f-259a-4dc0-80da-ee1a8551c836 service swift] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:55:22.855933 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10503438-3617-48d3-8d6a-4dcdc3157abe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8fd0f6e5b144d2ca78b8d96722502a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:55:22.861700 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10503438-3617-48d3-8d6a-4dcdc3157abe service placement] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:22.862272 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10503438-3617-48d3-8d6a-4dcdc3157abe service placement] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:22.862697 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10503438-3617-48d3-8d6a-4dcdc3157abe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:22.865149 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-10503438-3617-48d3-8d6a-4dcdc3157abe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:55:22.883420 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10503438-3617-48d3-8d6a-4dcdc3157abe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:55:22.887697 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10503438-3617-48d3-8d6a-4dcdc3157abe service placement] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:55:22.908175 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1306/2581] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:55:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:55:22.968432 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a341f80-6e3c-48e1-9e01-e7fdb7975377 service neutron] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:55:22.970589 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-8a341f80-6e3c-48e1-9e01-e7fdb7975377 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:55:22.990519 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a341f80-6e3c-48e1-9e01-e7fdb7975377 service neutron] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:55:23.001927 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9012e8-53da-4507-828d-0bb8082c0cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:55:23.003050 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9012e8-53da-4507-828d-0bb8082c0cd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:23.003668 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9012e8-53da-4507-828d-0bb8082c0cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:23.004152 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9012e8-53da-4507-828d-0bb8082c0cd5 admin admin] PATH_INFO: `/v3/users/bcd83a222eb04368a409db06389f1b7d` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:23.005895 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9012e8-53da-4507-828d-0bb8082c0cd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcd83a222eb04368a409db06389f1b7d)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:55:23.009639 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9012e8-53da-4507-828d-0bb8082c0cd5 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:55:23.240762 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635b36e-bea1-4314-bd1c-9a7c25037392 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:23.241089 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635b36e-bea1-4314-bd1c-9a7c25037392 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:23.241320 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635b36e-bea1-4314-bd1c-9a7c25037392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:23.275356 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-d635b36e-bea1-4314-bd1c-9a7c25037392 None None] MFA Rules not processed for user `df23a24f4d5948deb641de1cefad1f69`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:55:23.277185 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-d635b36e-bea1-4314-bd1c-9a7c25037392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:55:23.325131 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-2d9012e8-53da-4507-828d-0bb8082c0cd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcd83a222eb04368a409db06389f1b7d'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:23.365844 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-2d9012e8-53da-4507-828d-0bb8082c0cd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcd83a222eb04368a409db06389f1b7d'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:23.380782 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1307/2582] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:55:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:55:23.387013 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d635b36e-bea1-4314-bd1c-9a7c25037392 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:55:23.389676 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-2d9012e8-53da-4507-828d-0bb8082c0cd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcd83a222eb04368a409db06389f1b7d'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:23.400659 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d635b36e-bea1-4314-bd1c-9a7c25037392 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:55:23.407579 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6823144d-a964-4484-a562-0f375c3c3ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:55:23.416031 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6823144d-a964-4484-a562-0f375c3c3ec5 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:23.416031 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6823144d-a964-4484-a562-0f375c3c3ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:23.416031 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6823144d-a964-4484-a562-0f375c3c3ec5 admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:23.416031 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6823144d-a964-4484-a562-0f375c3c3ec5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:55:23.416031 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6823144d-a964-4484-a562-0f375c3c3ec5 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:55:23.480778 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-2d9012e8-53da-4507-828d-0bb8082c0cd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcd83a222eb04368a409db06389f1b7d'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:23.481177 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-2d9012e8-53da-4507-828d-0bb8082c0cd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcd83a222eb04368a409db06389f1b7d'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:23.490324 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1276/2583] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:55:22 2022] DELETE /identity/v3/users/bcd83a222eb04368a409db06389f1b7d => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:55:23.603069 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1308/2584] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:55:23 2022] POST /identity/v3/projects => generated 377 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:55:23.607680 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9012e8-53da-4507-828d-0bb8082c0cd5 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:55:23.613650 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-2d9012e8-53da-4507-828d-0bb8082c0cd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:55:23.617151 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:55:23.687013 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9012e8-53da-4507-828d-0bb8082c0cd5 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:55:23.767723 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf02f201-a33d-4881-b59d-121414e4b5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:55:23.768424 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf02f201-a33d-4881-b59d-121414e4b5b1 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:23.768691 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf02f201-a33d-4881-b59d-121414e4b5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:23.768997 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf02f201-a33d-4881-b59d-121414e4b5b1 admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:23.770665 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf02f201-a33d-4881-b59d-121414e4b5b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:55:23.773433 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf02f201-a33d-4881-b59d-121414e4b5b1 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:55:23.819389 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6823144d-a964-4484-a562-0f375c3c3ec5 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:55:23.822453 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-6823144d-a964-4484-a562-0f375c3c3ec5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:55:23.854308 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6823144d-a964-4484-a562-0f375c3c3ec5 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:55:23.860038 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd9904-4e4d-4719-8254-e0e81a0d5a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:55:23.861129 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd9904-4e4d-4719-8254-e0e81a0d5a40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:23.861471 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd9904-4e4d-4719-8254-e0e81a0d5a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:23.862021 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd9904-4e4d-4719-8254-e0e81a0d5a40 admin admin] PATH_INFO: `/v3/projects/d106ca51e01f439f82fd71df4dc2ed71` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:23.863439 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddd9904-4e4d-4719-8254-e0e81a0d5a40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d106ca51e01f439f82fd71df4dc2ed71)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:55:23.867410 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddd9904-4e4d-4719-8254-e0e81a0d5a40 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:55:23.903752 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:55:23.914683 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1277/2585] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:55:23 2022] POST /identity/v3/users => generated 295 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:55:23.919357 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf02f201-a33d-4881-b59d-121414e4b5b1 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:55:23.935545 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf02f201-a33d-4881-b59d-121414e4b5b1 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:55:23.947021 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9794f0d4-2625-42d6-899c-7e11316360ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:55:23.947668 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9794f0d4-2625-42d6-899c-7e11316360ef admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:23.948025 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9794f0d4-2625-42d6-899c-7e11316360ef admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:23.952905 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9794f0d4-2625-42d6-899c-7e11316360ef admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:23.952905 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9794f0d4-2625-42d6-899c-7e11316360ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:55:23.952905 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9794f0d4-2625-42d6-899c-7e11316360ef admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:55:23.969112 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1278/2586] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:55:23.973598 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9794f0d4-2625-42d6-899c-7e11316360ef admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:55:23.990631 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9794f0d4-2625-42d6-899c-7e11316360ef admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:55:24.001393 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-9ddd9904-4e4d-4719-8254-e0e81a0d5a40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd106ca51e01f439f82fd71df4dc2ed71', 'request_id': 'req-9ddd9904-4e4d-4719-8254-e0e81a0d5a40'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:24.008445 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-9ddd9904-4e4d-4719-8254-e0e81a0d5a40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd106ca51e01f439f82fd71df4dc2ed71', 'request_id': 'req-9ddd9904-4e4d-4719-8254-e0e81a0d5a40'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:24.018355 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-9ddd9904-4e4d-4719-8254-e0e81a0d5a40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd106ca51e01f439f82fd71df4dc2ed71', 'request_id': 'req-9ddd9904-4e4d-4719-8254-e0e81a0d5a40'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:24.066164 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd4866a-89d0-4fe0-afd2-8ae307294835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:55:24.067394 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4866a-89d0-4fe0-afd2-8ae307294835 admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:24.067609 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4866a-89d0-4fe0-afd2-8ae307294835 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:24.067989 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4866a-89d0-4fe0-afd2-8ae307294835 admin admin] PATH_INFO: `/v3/projects/e631bf7db91048f88772928b60dec182/users/b2a7c52332be42f19d9a2130b3f0caec/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:24.088088 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd4866a-89d0-4fe0-afd2-8ae307294835 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e631bf7db91048f88772928b60dec182, user_id=b2a7c52332be42f19d9a2130b3f0caec, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:55:24.091051 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd4866a-89d0-4fe0-afd2-8ae307294835 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:55:24.130567 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-9ddd9904-4e4d-4719-8254-e0e81a0d5a40 admin admin] The token cache is being invalidate because project d106ca51e01f439f82fd71df4dc2ed71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:55:24.130916 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-9ddd9904-4e4d-4719-8254-e0e81a0d5a40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:24.136200 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-9ddd9904-4e4d-4719-8254-e0e81a0d5a40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:24.136200 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1309/2587] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:55:23 2022] DELETE /identity/v3/projects/d106ca51e01f439f82fd71df4dc2ed71 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:55:24.138582 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632550c5-4e68-405b-8c9c-32f2442eada2 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:24.138945 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632550c5-4e68-405b-8c9c-32f2442eada2 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:24.139234 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632550c5-4e68-405b-8c9c-32f2442eada2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:24.161081 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-632550c5-4e68-405b-8c9c-32f2442eada2 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:55:24.162569 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-632550c5-4e68-405b-8c9c-32f2442eada2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:55:24.172757 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1310/2588] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:55:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:55:24.179681 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c07b1-a29f-4827-b786-41778a011509 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:24.180069 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c07b1-a29f-4827-b786-41778a011509 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:24.180453 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c07b1-a29f-4827-b786-41778a011509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:24.203126 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-5f5c07b1-a29f-4827-b786-41778a011509 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:55:24.204362 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-5f5c07b1-a29f-4827-b786-41778a011509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:55:24.213032 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1311/2589] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:55:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:55:24.217789 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c07b1-a29f-4827-b786-41778a011509 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:55:24.228548 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c07b1-a29f-4827-b786-41778a011509 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:55:24.237869 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1279/2590] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:55:23 2022] PUT /identity/v3/projects/e631bf7db91048f88772928b60dec182/users/b2a7c52332be42f19d9a2130b3f0caec/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:55:24.239572 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd4866a-89d0-4fe0-afd2-8ae307294835 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:55:24.240841 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-bbd4866a-89d0-4fe0-afd2-8ae307294835 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:55:24.262466 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:55:24.319206 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708d91e6-a3f8-47f6-9f8d-ab80f8e49f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:55:24.319955 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d91e6-a3f8-47f6-9f8d-ab80f8e49f95 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:24.320922 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d91e6-a3f8-47f6-9f8d-ab80f8e49f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:24.323640 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d91e6-a3f8-47f6-9f8d-ab80f8e49f95 admin admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:24.323640 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708d91e6-a3f8-47f6-9f8d-ab80f8e49f95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:55:24.323640 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708d91e6-a3f8-47f6-9f8d-ab80f8e49f95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:55:24.324831 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708d91e6-a3f8-47f6-9f8d-ab80f8e49f95 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:55:24.339423 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1312/2591] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:55:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:55:24.344110 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708d91e6-a3f8-47f6-9f8d-ab80f8e49f95 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:55:24.353411 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd4866a-89d0-4fe0-afd2-8ae307294835 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:55:24.355245 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708d91e6-a3f8-47f6-9f8d-ab80f8e49f95 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:55:24.360707 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da812f39-b413-4197-8de6-6764b191fa13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:55:24.363667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da812f39-b413-4197-8de6-6764b191fa13 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:24.363667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da812f39-b413-4197-8de6-6764b191fa13 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:24.363667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da812f39-b413-4197-8de6-6764b191fa13 admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:24.363667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da812f39-b413-4197-8de6-6764b191fa13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:55:24.364037 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f3464-4ae6-4726-9cad-e1697230e890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:55:24.364565 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f3464-4ae6-4726-9cad-e1697230e890 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:24.364901 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f3464-4ae6-4726-9cad-e1697230e890 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:24.365199 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f3464-4ae6-4726-9cad-e1697230e890 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:24.365857 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4f3464-4ae6-4726-9cad-e1697230e890 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:55:24.366772 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da812f39-b413-4197-8de6-6764b191fa13 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:55:24.368683 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4f3464-4ae6-4726-9cad-e1697230e890 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:55:24.385091 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1280/2592] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:55:24 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:55:24.387096 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f3464-4ae6-4726-9cad-e1697230e890 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:55:24.397576 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f3464-4ae6-4726-9cad-e1697230e890 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:55:24.405813 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77761da-395a-4da1-92b0-8ef0d393f36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:55:24.406518 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77761da-395a-4da1-92b0-8ef0d393f36f admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:24.406806 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77761da-395a-4da1-92b0-8ef0d393f36f admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:24.407085 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77761da-395a-4da1-92b0-8ef0d393f36f admin admin] PATH_INFO: `/v3/projects/e631bf7db91048f88772928b60dec182/users/b2a7c52332be42f19d9a2130b3f0caec/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:24.409253 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77761da-395a-4da1-92b0-8ef0d393f36f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e631bf7db91048f88772928b60dec182, user_id=b2a7c52332be42f19d9a2130b3f0caec, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:55:24.412006 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77761da-395a-4da1-92b0-8ef0d393f36f admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:55:24.423813 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1281/2593] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:55:24 2022] PUT /identity/v3/projects/e631bf7db91048f88772928b60dec182/users/b2a7c52332be42f19d9a2130b3f0caec/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:55:24.522797 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1313/2594] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:55:24 2022] POST /identity/v3/projects => generated 385 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:55:24.528019 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77761da-395a-4da1-92b0-8ef0d393f36f admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:55:24.605531 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77761da-395a-4da1-92b0-8ef0d393f36f admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:55:24.679708 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a890c8-7d82-428e-8b75-ac789c6b089c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:55:24.680284 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a890c8-7d82-428e-8b75-ac789c6b089c admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:24.681128 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a890c8-7d82-428e-8b75-ac789c6b089c admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:24.681406 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a890c8-7d82-428e-8b75-ac789c6b089c admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:24.682143 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a890c8-7d82-428e-8b75-ac789c6b089c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:55:24.684079 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a890c8-7d82-428e-8b75-ac789c6b089c admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:55:24.956577 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1282/2595] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:55:24 2022] POST /identity/v3/users => generated 293 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:55:24.958292 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da812f39-b413-4197-8de6-6764b191fa13 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:55:24.972882 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da812f39-b413-4197-8de6-6764b191fa13 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:55:24.995087 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206da4b-32f1-4f12-bbd1-ee2b22f35b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:55:24.996971 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206da4b-32f1-4f12-bbd1-ee2b22f35b25 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:24.997722 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206da4b-32f1-4f12-bbd1-ee2b22f35b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:24.998568 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206da4b-32f1-4f12-bbd1-ee2b22f35b25 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:25.001096 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e206da4b-32f1-4f12-bbd1-ee2b22f35b25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:55:25.003205 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e206da4b-32f1-4f12-bbd1-ee2b22f35b25 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:55:25.027178 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1314/2596] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:55:24 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:55:25.031778 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a890c8-7d82-428e-8b75-ac789c6b089c admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:55:25.044948 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a890c8-7d82-428e-8b75-ac789c6b089c admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:55:25.050561 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e97c00-1e56-4e96-8ea8-8f4d11b3dcba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:55:25.051497 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e97c00-1e56-4e96-8ea8-8f4d11b3dcba admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:25.052043 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e97c00-1e56-4e96-8ea8-8f4d11b3dcba admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:25.052586 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e97c00-1e56-4e96-8ea8-8f4d11b3dcba admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:25.053798 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e97c00-1e56-4e96-8ea8-8f4d11b3dcba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:55:25.056134 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e97c00-1e56-4e96-8ea8-8f4d11b3dcba admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:55:25.068150 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1283/2597] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:55:25.073471 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206da4b-32f1-4f12-bbd1-ee2b22f35b25 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:55:25.085208 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206da4b-32f1-4f12-bbd1-ee2b22f35b25 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:55:25.092011 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae745665-61ed-4f8f-9a41-a4005cf6c50b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:55:25.092675 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae745665-61ed-4f8f-9a41-a4005cf6c50b admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:25.096524 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae745665-61ed-4f8f-9a41-a4005cf6c50b admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:25.096524 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae745665-61ed-4f8f-9a41-a4005cf6c50b admin admin] PATH_INFO: `/v3/projects/77495a957104457c8f0cbee8c01a4536/users/f3121e61f02e4869b8214a5caf1b580a/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:25.111102 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae745665-61ed-4f8f-9a41-a4005cf6c50b admin admin] RBAC: Authorizing `identity:create_grant(project_id=77495a957104457c8f0cbee8c01a4536, user_id=f3121e61f02e4869b8214a5caf1b580a, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:55:25.114153 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae745665-61ed-4f8f-9a41-a4005cf6c50b admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:55:25.195149 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1315/2598] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:55:25 2022] PUT /identity/v3/projects/77495a957104457c8f0cbee8c01a4536/users/f3121e61f02e4869b8214a5caf1b580a/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:55:25.200269 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2727a-77f3-45b3-ba1e-ef0bc66cb216 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:25.200557 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2727a-77f3-45b3-ba1e-ef0bc66cb216 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:25.200876 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2727a-77f3-45b3-ba1e-ef0bc66cb216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:25.223490 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-def2727a-77f3-45b3-ba1e-ef0bc66cb216 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:55:25.229190 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-def2727a-77f3-45b3-ba1e-ef0bc66cb216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:55:25.244319 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:55:25.299360 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1284/2599] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:55:25.315556 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae745665-61ed-4f8f-9a41-a4005cf6c50b admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:55:25.315556 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-ae745665-61ed-4f8f-9a41-a4005cf6c50b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:55:25.386517 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:55:25.417556 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae745665-61ed-4f8f-9a41-a4005cf6c50b admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:55:25.423100 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deabdd29-b263-4f1a-a885-a7581f890c3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:55:25.423947 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabdd29-b263-4f1a-a885-a7581f890c3d service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:25.424411 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabdd29-b263-4f1a-a885-a7581f890c3d service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:25.424864 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabdd29-b263-4f1a-a885-a7581f890c3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:25.455141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deabdd29-b263-4f1a-a885-a7581f890c3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:55:25.461002 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deabdd29-b263-4f1a-a885-a7581f890c3d service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:55:25.480038 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1316/2600] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:55:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:55:25.734751 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def2727a-77f3-45b3-ba1e-ef0bc66cb216 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:55:25.737390 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-def2727a-77f3-45b3-ba1e-ef0bc66cb216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:55:25.757735 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def2727a-77f3-45b3-ba1e-ef0bc66cb216 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:55:25.766019 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb978f-26ab-413b-87cb-e3d4d6bf6bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:55:25.777037 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb978f-26ab-413b-87cb-e3d4d6bf6bad admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:25.777037 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb978f-26ab-413b-87cb-e3d4d6bf6bad admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:25.777037 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb978f-26ab-413b-87cb-e3d4d6bf6bad admin admin] PATH_INFO: `/v3/users/8cad1ee24ee84aa18da56d1145eac98a` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:25.777037 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bb978f-26ab-413b-87cb-e3d4d6bf6bad admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cad1ee24ee84aa18da56d1145eac98a)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:55:25.777037 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bb978f-26ab-413b-87cb-e3d4d6bf6bad admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:55:26.163899 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-c8bb978f-26ab-413b-87cb-e3d4d6bf6bad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cad1ee24ee84aa18da56d1145eac98a'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:26.181128 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-c8bb978f-26ab-413b-87cb-e3d4d6bf6bad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cad1ee24ee84aa18da56d1145eac98a'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:26.192099 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-c8bb978f-26ab-413b-87cb-e3d4d6bf6bad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cad1ee24ee84aa18da56d1145eac98a'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:26.279418 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-c8bb978f-26ab-413b-87cb-e3d4d6bf6bad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cad1ee24ee84aa18da56d1145eac98a'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:26.280515 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-c8bb978f-26ab-413b-87cb-e3d4d6bf6bad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cad1ee24ee84aa18da56d1145eac98a'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:26.287937 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1285/2601] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:55:25 2022] DELETE /identity/v3/users/8cad1ee24ee84aa18da56d1145eac98a => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:55:26.563494 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deabdd29-b263-4f1a-a885-a7581f890c3d service neutron] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:55:26.574949 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deabdd29-b263-4f1a-a885-a7581f890c3d service neutron] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:55:26.652874 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6350d4-331d-47d1-9559-6e5dcb1b622d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:55:26.653534 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6350d4-331d-47d1-9559-6e5dcb1b622d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:26.653821 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6350d4-331d-47d1-9559-6e5dcb1b622d admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:26.654098 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6350d4-331d-47d1-9559-6e5dcb1b622d admin admin] PATH_INFO: `/v3/projects/6468612f8d4e4172a8bcc16f0ba73e5a` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:26.655588 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6350d4-331d-47d1-9559-6e5dcb1b622d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6468612f8d4e4172a8bcc16f0ba73e5a)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:55:26.658756 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6350d4-331d-47d1-9559-6e5dcb1b622d admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:55:26.844032 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ca6350d4-331d-47d1-9559-6e5dcb1b622d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6468612f8d4e4172a8bcc16f0ba73e5a', 'request_id': 'req-ca6350d4-331d-47d1-9559-6e5dcb1b622d'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:26.855072 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ca6350d4-331d-47d1-9559-6e5dcb1b622d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6468612f8d4e4172a8bcc16f0ba73e5a', 'request_id': 'req-ca6350d4-331d-47d1-9559-6e5dcb1b622d'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:26.861891 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ca6350d4-331d-47d1-9559-6e5dcb1b622d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6468612f8d4e4172a8bcc16f0ba73e5a', 'request_id': 'req-ca6350d4-331d-47d1-9559-6e5dcb1b622d'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:26.979661 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ca6350d4-331d-47d1-9559-6e5dcb1b622d admin admin] The token cache is being invalidate because project 6468612f8d4e4172a8bcc16f0ba73e5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:55:26.980217 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ca6350d4-331d-47d1-9559-6e5dcb1b622d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:26.981091 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ca6350d4-331d-47d1-9559-6e5dcb1b622d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:26.983167 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1317/2602] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:55:26 2022] DELETE /identity/v3/projects/6468612f8d4e4172a8bcc16f0ba73e5a => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:55:26.988280 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb978f-26ab-413b-87cb-e3d4d6bf6bad admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:55:26.989281 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-c8bb978f-26ab-413b-87cb-e3d4d6bf6bad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:55:27.026410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb978f-26ab-413b-87cb-e3d4d6bf6bad admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:55:27.112546 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bb3573-a6f7-4665-a221-221ef71583c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:55:27.113975 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb3573-a6f7-4665-a221-221ef71583c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:27.114912 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb3573-a6f7-4665-a221-221ef71583c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:27.115823 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb3573-a6f7-4665-a221-221ef71583c1 admin admin] PATH_INFO: `/v3/users/99078ca3f1604e169781f89ac8ca5182` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:27.118749 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bb3573-a6f7-4665-a221-221ef71583c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99078ca3f1604e169781f89ac8ca5182)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:55:27.121307 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bb3573-a6f7-4665-a221-221ef71583c1 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:55:27.378851 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:55:27.388213 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-c6bb3573-a6f7-4665-a221-221ef71583c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99078ca3f1604e169781f89ac8ca5182'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:27.401039 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-c6bb3573-a6f7-4665-a221-221ef71583c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99078ca3f1604e169781f89ac8ca5182'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:27.409551 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-c6bb3573-a6f7-4665-a221-221ef71583c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99078ca3f1604e169781f89ac8ca5182'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:27.525848 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-c6bb3573-a6f7-4665-a221-221ef71583c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99078ca3f1604e169781f89ac8ca5182'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:27.530012 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-c6bb3573-a6f7-4665-a221-221ef71583c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99078ca3f1604e169781f89ac8ca5182'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:27.533806 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1286/2603] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:55:26 2022] DELETE /identity/v3/users/99078ca3f1604e169781f89ac8ca5182 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:55:27.818585 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6350d4-331d-47d1-9559-6e5dcb1b622d admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:55:27.822477 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-ca6350d4-331d-47d1-9559-6e5dcb1b622d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:55:27.842245 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6350d4-331d-47d1-9559-6e5dcb1b622d admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:55:27.930124 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a7bed-9e30-47d6-840a-3cda273fa543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:55:27.930975 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a7bed-9e30-47d6-840a-3cda273fa543 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:27.931478 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a7bed-9e30-47d6-840a-3cda273fa543 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:27.942248 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a7bed-9e30-47d6-840a-3cda273fa543 admin admin] PATH_INFO: `/v3/projects/550e5821300144f68160dd9a48f9ef2d` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:27.942248 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328a7bed-9e30-47d6-840a-3cda273fa543 admin admin] RBAC: Authorizing `identity:delete_project(project_id=550e5821300144f68160dd9a48f9ef2d)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:55:27.942248 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328a7bed-9e30-47d6-840a-3cda273fa543 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:55:27.957635 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:55:28.082440 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-328a7bed-9e30-47d6-840a-3cda273fa543 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '550e5821300144f68160dd9a48f9ef2d', 'request_id': 'req-328a7bed-9e30-47d6-840a-3cda273fa543'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:28.098381 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-328a7bed-9e30-47d6-840a-3cda273fa543 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '550e5821300144f68160dd9a48f9ef2d', 'request_id': 'req-328a7bed-9e30-47d6-840a-3cda273fa543'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:28.108216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-328a7bed-9e30-47d6-840a-3cda273fa543 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '550e5821300144f68160dd9a48f9ef2d', 'request_id': 'req-328a7bed-9e30-47d6-840a-3cda273fa543'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:28.175638 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-328a7bed-9e30-47d6-840a-3cda273fa543 admin admin] The token cache is being invalidate because project 550e5821300144f68160dd9a48f9ef2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:55:28.176191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-328a7bed-9e30-47d6-840a-3cda273fa543 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:28.178930 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-328a7bed-9e30-47d6-840a-3cda273fa543 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:55:28.182251 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1318/2604] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:55:27 2022] DELETE /identity/v3/projects/550e5821300144f68160dd9a48f9ef2d => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:55:32.495554 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61994129-2651-4216-a91c-1f5b1740a487 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:32.496466 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61994129-2651-4216-a91c-1f5b1740a487 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:32.496882 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61994129-2651-4216-a91c-1f5b1740a487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:32.562928 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-61994129-2651-4216-a91c-1f5b1740a487 None None] MFA Rules not processed for user `b2a7c52332be42f19d9a2130b3f0caec`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:55:32.562928 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-61994129-2651-4216-a91c-1f5b1740a487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:55:32.658394 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1287/2605] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:55:32 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:55:32.675698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a7bed-9e30-47d6-840a-3cda273fa543 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:55:32.677445 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-328a7bed-9e30-47d6-840a-3cda273fa543 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:55:32.811455 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a7bed-9e30-47d6-840a-3cda273fa543 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:55:32.817904 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a88df87-53f4-402b-9df9-58447c16fb03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:55:32.818656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88df87-53f4-402b-9df9-58447c16fb03 service nova] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:32.818941 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88df87-53f4-402b-9df9-58447c16fb03 service nova] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:32.819265 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88df87-53f4-402b-9df9-58447c16fb03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:32.822228 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-1a88df87-53f4-402b-9df9-58447c16fb03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:55:32.950704 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a88df87-53f4-402b-9df9-58447c16fb03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:55:32.953319 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a88df87-53f4-402b-9df9-58447c16fb03 service nova] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:55:32.975537 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1319/2606] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:55:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:55:33.221038 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd821e-aba9-4717-bc56-d4fc89e21c39 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:33.222592 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd821e-aba9-4717-bc56-d4fc89e21c39 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:33.222951 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd821e-aba9-4717-bc56-d4fc89e21c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:33.260959 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-95bd821e-aba9-4717-bc56-d4fc89e21c39 None None] MFA Rules not processed for user `f3121e61f02e4869b8214a5caf1b580a`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:55:33.263768 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-95bd821e-aba9-4717-bc56-d4fc89e21c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:55:33.361820 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1288/2607] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:55:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:55:33.384864 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a88df87-53f4-402b-9df9-58447c16fb03 service nova] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:55:33.386937 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-1a88df87-53f4-402b-9df9-58447c16fb03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:55:33.415996 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a88df87-53f4-402b-9df9-58447c16fb03 service nova] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:55:33.423405 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1c63e8-b007-418a-9e10-888b94094f00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:55:33.423405 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c63e8-b007-418a-9e10-888b94094f00 service nova] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:33.423857 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c63e8-b007-418a-9e10-888b94094f00 service nova] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:33.423940 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c63e8-b007-418a-9e10-888b94094f00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:33.433671 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1c63e8-b007-418a-9e10-888b94094f00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:55:33.436658 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1c63e8-b007-418a-9e10-888b94094f00 service nova] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:55:33.457036 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1320/2608] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:55:33 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:55:36.567496 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c5731-29b6-46e0-8f76-f19ca7084bf6 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:36.568501 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c5731-29b6-46e0-8f76-f19ca7084bf6 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:36.568817 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c5731-29b6-46e0-8f76-f19ca7084bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:36.597713 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-748c5731-29b6-46e0-8f76-f19ca7084bf6 None None] MFA Rules not processed for user `f3121e61f02e4869b8214a5caf1b580a`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:55:36.599274 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-748c5731-29b6-46e0-8f76-f19ca7084bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:55:36.608118 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1289/2609] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:55:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:55:43.442759 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=82,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:55:46.584122 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=84,keystone:DELETE=5,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:55:54.273523 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2807d1-e36f-41a7-a1b2-5690ca34b634 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:54.274004 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2807d1-e36f-41a7-a1b2-5690ca34b634 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:54.274127 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2807d1-e36f-41a7-a1b2-5690ca34b634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:54.295331 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-9f2807d1-e36f-41a7-a1b2-5690ca34b634 None None] MFA Rules not processed for user `df23a24f4d5948deb641de1cefad1f69`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:55:54.296535 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-9f2807d1-e36f-41a7-a1b2-5690ca34b634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:55:54.303746 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1321/2610] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:55:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:55:59.626540 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c5731-29b6-46e0-8f76-f19ca7084bf6 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:55:59.626540 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-748c5731-29b6-46e0-8f76-f19ca7084bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:55:59.690599 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c5731-29b6-46e0-8f76-f19ca7084bf6 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:55:59.694206 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489446c1-3911-48ab-af72-b0218154ca1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:55:59.694817 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489446c1-3911-48ab-af72-b0218154ca1b service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:59.695108 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489446c1-3911-48ab-af72-b0218154ca1b service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:59.695362 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489446c1-3911-48ab-af72-b0218154ca1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:59.697100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-489446c1-3911-48ab-af72-b0218154ca1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:55:59.765589 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489446c1-3911-48ab-af72-b0218154ca1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:55:59.767093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489446c1-3911-48ab-af72-b0218154ca1b service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:55:59.779814 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1290/2611] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:55:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:56:04.080765 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2807d1-e36f-41a7-a1b2-5690ca34b634 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:04.082244 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-9f2807d1-e36f-41a7-a1b2-5690ca34b634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:04.097975 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2807d1-e36f-41a7-a1b2-5690ca34b634 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:04.102616 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c6cfc-ba76-44c2-b811-fe25187e33ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:04.103177 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c6cfc-ba76-44c2-b811-fe25187e33ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:04.103451 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c6cfc-ba76-44c2-b811-fe25187e33ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:04.103809 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c6cfc-ba76-44c2-b811-fe25187e33ae admin admin] PATH_INFO: `/v3/users/38a89946e1fb4b43aed992078dba0b4f` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:04.104991 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c6cfc-ba76-44c2-b811-fe25187e33ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=38a89946e1fb4b43aed992078dba0b4f)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:04.106870 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c6cfc-ba76-44c2-b811-fe25187e33ae admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:04.252249 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-a39c6cfc-ba76-44c2-b811-fe25187e33ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38a89946e1fb4b43aed992078dba0b4f'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:04.329762 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-a39c6cfc-ba76-44c2-b811-fe25187e33ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38a89946e1fb4b43aed992078dba0b4f'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:04.343390 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-a39c6cfc-ba76-44c2-b811-fe25187e33ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38a89946e1fb4b43aed992078dba0b4f'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:04.350290 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-a39c6cfc-ba76-44c2-b811-fe25187e33ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38a89946e1fb4b43aed992078dba0b4f'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:04.350993 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-a39c6cfc-ba76-44c2-b811-fe25187e33ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38a89946e1fb4b43aed992078dba0b4f'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:04.357332 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1322/2612] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:56:04 2022] DELETE /identity/v3/users/38a89946e1fb4b43aed992078dba0b4f => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:56:04.612233 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489446c1-3911-48ab-af72-b0218154ca1b service neutron] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:04.626039 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489446c1-3911-48ab-af72-b0218154ca1b service neutron] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:04.688247 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea2a49a-93b3-4d7d-a2c9-4c2adcccc0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:04.688929 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2a49a-93b3-4d7d-a2c9-4c2adcccc0d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:04.689276 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2a49a-93b3-4d7d-a2c9-4c2adcccc0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:04.689557 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2a49a-93b3-4d7d-a2c9-4c2adcccc0d9 admin admin] PATH_INFO: `/v3/projects/a0e8b532701e43a39a0fd260e3e5d12d` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:04.690701 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea2a49a-93b3-4d7d-a2c9-4c2adcccc0d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0e8b532701e43a39a0fd260e3e5d12d)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:04.692613 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea2a49a-93b3-4d7d-a2c9-4c2adcccc0d9 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:04.837844 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-fea2a49a-93b3-4d7d-a2c9-4c2adcccc0d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0e8b532701e43a39a0fd260e3e5d12d', 'request_id': 'req-fea2a49a-93b3-4d7d-a2c9-4c2adcccc0d9'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:04.840689 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:56:04.951821 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-fea2a49a-93b3-4d7d-a2c9-4c2adcccc0d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0e8b532701e43a39a0fd260e3e5d12d', 'request_id': 'req-fea2a49a-93b3-4d7d-a2c9-4c2adcccc0d9'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:04.959799 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-fea2a49a-93b3-4d7d-a2c9-4c2adcccc0d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0e8b532701e43a39a0fd260e3e5d12d', 'request_id': 'req-fea2a49a-93b3-4d7d-a2c9-4c2adcccc0d9'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:04.983431 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-fea2a49a-93b3-4d7d-a2c9-4c2adcccc0d9 admin admin] The token cache is being invalidate because project a0e8b532701e43a39a0fd260e3e5d12d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:56:04.983737 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-fea2a49a-93b3-4d7d-a2c9-4c2adcccc0d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:04.985080 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-fea2a49a-93b3-4d7d-a2c9-4c2adcccc0d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:04.990229 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1291/2613] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:56:04 2022] DELETE /identity/v3/projects/a0e8b532701e43a39a0fd260e3e5d12d => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:56:04.992311 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e246d9a-c0de-440f-8abc-bec54f8a5caf None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:04.992311 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e246d9a-c0de-440f-8abc-bec54f8a5caf None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:04.992557 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e246d9a-c0de-440f-8abc-bec54f8a5caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:05.017742 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-9e246d9a-c0de-440f-8abc-bec54f8a5caf None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:56:05.018926 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-9e246d9a-c0de-440f-8abc-bec54f8a5caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:05.087404 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1323/2614] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:56:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:56:05.095714 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c40aa-fcd7-4097-a632-cd10613b41db None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:05.096004 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c40aa-fcd7-4097-a632-cd10613b41db None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:05.096318 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c40aa-fcd7-4097-a632-cd10613b41db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:05.121273 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-1b7c40aa-fcd7-4097-a632-cd10613b41db None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:56:05.121273 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-1b7c40aa-fcd7-4097-a632-cd10613b41db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:05.127785 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1292/2615] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:56:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:56:05.132004 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e246d9a-c0de-440f-8abc-bec54f8a5caf None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:05.142235 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e246d9a-c0de-440f-8abc-bec54f8a5caf None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:05.146890 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5480ca-55a4-4633-975f-9583bab6adf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:05.147481 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5480ca-55a4-4633-975f-9583bab6adf5 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:05.147757 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5480ca-55a4-4633-975f-9583bab6adf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:05.148027 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5480ca-55a4-4633-975f-9583bab6adf5 admin admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:05.148664 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5480ca-55a4-4633-975f-9583bab6adf5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:56:05.148992 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5480ca-55a4-4633-975f-9583bab6adf5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:05.150790 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5480ca-55a4-4633-975f-9583bab6adf5 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:05.162222 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1324/2616] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:56:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:56:05.169656 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7c40aa-fcd7-4097-a632-cd10613b41db None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:05.178826 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7c40aa-fcd7-4097-a632-cd10613b41db None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:05.184151 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b61ffd-deeb-4e4a-ac02-25355b7a8868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:05.184678 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b61ffd-deeb-4e4a-ac02-25355b7a8868 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:05.185057 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b61ffd-deeb-4e4a-ac02-25355b7a8868 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:05.185331 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b61ffd-deeb-4e4a-ac02-25355b7a8868 admin admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:05.185964 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b61ffd-deeb-4e4a-ac02-25355b7a8868 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:05.187857 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b61ffd-deeb-4e4a-ac02-25355b7a8868 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:05.317615 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1293/2617] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:56:05 2022] POST /identity/v3/projects => generated 363 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:56:05.327795 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5480ca-55a4-4633-975f-9583bab6adf5 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:05.339588 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5480ca-55a4-4633-975f-9583bab6adf5 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:05.388890 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:56:05.397628 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a0325-e926-4d09-b379-bb0ffd376fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:05.398235 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a0325-e926-4d09-b379-bb0ffd376fc8 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:05.398517 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a0325-e926-4d09-b379-bb0ffd376fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:05.398791 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a0325-e926-4d09-b379-bb0ffd376fc8 admin admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:05.399413 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016a0325-e926-4d09-b379-bb0ffd376fc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:05.402170 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016a0325-e926-4d09-b379-bb0ffd376fc8 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:05.553597 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1325/2618] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:56:05 2022] POST /identity/v3/users => generated 282 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:56:05.557816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b61ffd-deeb-4e4a-ac02-25355b7a8868 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:05.568128 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b61ffd-deeb-4e4a-ac02-25355b7a8868 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:05.574593 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819cce16-3340-4b59-9c42-811e7aa8b4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:05.574593 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819cce16-3340-4b59-9c42-811e7aa8b4ee admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:05.574593 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819cce16-3340-4b59-9c42-811e7aa8b4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:05.574968 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819cce16-3340-4b59-9c42-811e7aa8b4ee admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:05.576220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819cce16-3340-4b59-9c42-811e7aa8b4ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:05.577410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819cce16-3340-4b59-9c42-811e7aa8b4ee admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:05.587571 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1294/2619] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:56:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:56:05.592747 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a0325-e926-4d09-b379-bb0ffd376fc8 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:05.602841 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a0325-e926-4d09-b379-bb0ffd376fc8 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:05.607514 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45cb303-1062-4602-b36c-bc5415afded4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:05.608111 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45cb303-1062-4602-b36c-bc5415afded4 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:05.613632 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45cb303-1062-4602-b36c-bc5415afded4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:05.613632 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45cb303-1062-4602-b36c-bc5415afded4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:05.613632 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45cb303-1062-4602-b36c-bc5415afded4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:05.613632 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45cb303-1062-4602-b36c-bc5415afded4 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:05.621972 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1326/2620] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:56:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:56:05.626503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819cce16-3340-4b59-9c42-811e7aa8b4ee admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:05.638030 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819cce16-3340-4b59-9c42-811e7aa8b4ee admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:05.642788 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc251df-1318-4e1f-9fdc-9eaa20c489a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:05.643472 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc251df-1318-4e1f-9fdc-9eaa20c489a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:05.643746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc251df-1318-4e1f-9fdc-9eaa20c489a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:05.644016 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc251df-1318-4e1f-9fdc-9eaa20c489a5 admin admin] PATH_INFO: `/v3/projects/44142c9c52034f79b1f5faac920c3f49/users/30ebdbf5ab3d4cfda9bee871a930dccf/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:05.659832 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc251df-1318-4e1f-9fdc-9eaa20c489a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44142c9c52034f79b1f5faac920c3f49, user_id=30ebdbf5ab3d4cfda9bee871a930dccf, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:05.662517 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc251df-1318-4e1f-9fdc-9eaa20c489a5 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:05.765405 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1295/2621] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:56:05 2022] PUT /identity/v3/projects/44142c9c52034f79b1f5faac920c3f49/users/30ebdbf5ab3d4cfda9bee871a930dccf/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:56:05.776599 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6135bb92-1b9e-4be6-974b-eef9f675e9c0 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:05.776909 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6135bb92-1b9e-4be6-974b-eef9f675e9c0 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:05.777186 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6135bb92-1b9e-4be6-974b-eef9f675e9c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:05.807258 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-6135bb92-1b9e-4be6-974b-eef9f675e9c0 None None] MFA Rules not processed for user `30ebdbf5ab3d4cfda9bee871a930dccf`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:56:05.808350 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-6135bb92-1b9e-4be6-974b-eef9f675e9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:05.872320 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1327/2622] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:56:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:56:05.877044 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc251df-1318-4e1f-9fdc-9eaa20c489a5 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:05.940685 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc251df-1318-4e1f-9fdc-9eaa20c489a5 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:05.945411 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed77191a-4e4d-44a9-a319-4be022d6ccd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:05.946006 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed77191a-4e4d-44a9-a319-4be022d6ccd3 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:05.946287 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed77191a-4e4d-44a9-a319-4be022d6ccd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:05.946558 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed77191a-4e4d-44a9-a319-4be022d6ccd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:05.947344 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed77191a-4e4d-44a9-a319-4be022d6ccd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:05.949423 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed77191a-4e4d-44a9-a319-4be022d6ccd3 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:06.071610 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1296/2623] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:56:05 2022] POST /identity/v3/projects => generated 365 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:56:06.072810 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6135bb92-1b9e-4be6-974b-eef9f675e9c0 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:06.083634 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6135bb92-1b9e-4be6-974b-eef9f675e9c0 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:06.146482 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50403e64-510b-4754-90a3-4492469bec59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:06.147090 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50403e64-510b-4754-90a3-4492469bec59 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:06.147373 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50403e64-510b-4754-90a3-4492469bec59 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:06.147787 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50403e64-510b-4754-90a3-4492469bec59 admin admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:06.148437 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50403e64-510b-4754-90a3-4492469bec59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:06.151072 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50403e64-510b-4754-90a3-4492469bec59 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:06.256193 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1328/2624] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:56:06 2022] POST /identity/v3/users => generated 289 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:56:06.260262 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed77191a-4e4d-44a9-a319-4be022d6ccd3 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:06.271066 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed77191a-4e4d-44a9-a319-4be022d6ccd3 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:06.276191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae07252-331d-4eaf-b17a-72a7c8b7ed1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:06.276840 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae07252-331d-4eaf-b17a-72a7c8b7ed1b admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:06.277120 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae07252-331d-4eaf-b17a-72a7c8b7ed1b admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:06.277394 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae07252-331d-4eaf-b17a-72a7c8b7ed1b admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:06.278051 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae07252-331d-4eaf-b17a-72a7c8b7ed1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:06.279659 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae07252-331d-4eaf-b17a-72a7c8b7ed1b admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:06.292600 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1297/2625] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:56:06.296961 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50403e64-510b-4754-90a3-4492469bec59 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:06.308484 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50403e64-510b-4754-90a3-4492469bec59 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:06.313250 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a0584d-4251-4712-a8e5-987623080ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:06.313919 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0584d-4251-4712-a8e5-987623080ac7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:06.314217 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0584d-4251-4712-a8e5-987623080ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:06.317960 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0584d-4251-4712-a8e5-987623080ac7 admin admin] PATH_INFO: `/v3/projects/85cfdd48f9b242d890157f0a28cb0e40/users/7e0da36180ec477da9789cf6acebc58c/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:06.331559 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a0584d-4251-4712-a8e5-987623080ac7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85cfdd48f9b242d890157f0a28cb0e40, user_id=7e0da36180ec477da9789cf6acebc58c, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:06.334185 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a0584d-4251-4712-a8e5-987623080ac7 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:06.404254 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1329/2626] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:56:06 2022] PUT /identity/v3/projects/85cfdd48f9b242d890157f0a28cb0e40/users/7e0da36180ec477da9789cf6acebc58c/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:56:06.408359 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae07252-331d-4eaf-b17a-72a7c8b7ed1b admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:06.419888 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae07252-331d-4eaf-b17a-72a7c8b7ed1b admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:06.473286 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:56:06.484204 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ab08b-e047-4a93-a629-607ee39b87b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:06.484841 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ab08b-e047-4a93-a629-607ee39b87b1 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:06.485194 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ab08b-e047-4a93-a629-607ee39b87b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:06.485486 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ab08b-e047-4a93-a629-607ee39b87b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:06.486169 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33ab08b-e047-4a93-a629-607ee39b87b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:06.487827 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33ab08b-e047-4a93-a629-607ee39b87b1 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:06.498639 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1298/2627] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:56:06.503025 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a0584d-4251-4712-a8e5-987623080ac7 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:06.513584 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a0584d-4251-4712-a8e5-987623080ac7 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:06.518309 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d6d2fd-18d9-4184-9363-73649ea872bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:06.518949 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6d2fd-18d9-4184-9363-73649ea872bd admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:06.519228 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6d2fd-18d9-4184-9363-73649ea872bd admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:06.519503 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6d2fd-18d9-4184-9363-73649ea872bd admin admin] PATH_INFO: `/v3/projects/85cfdd48f9b242d890157f0a28cb0e40/users/7e0da36180ec477da9789cf6acebc58c/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:06.521557 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d6d2fd-18d9-4184-9363-73649ea872bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=85cfdd48f9b242d890157f0a28cb0e40, user_id=7e0da36180ec477da9789cf6acebc58c, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:06.524628 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d6d2fd-18d9-4184-9363-73649ea872bd admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:06.535706 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1330/2628] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:56:06 2022] PUT /identity/v3/projects/85cfdd48f9b242d890157f0a28cb0e40/users/7e0da36180ec477da9789cf6acebc58c/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:56:06.551428 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69f447-3e07-4d7b-a35e-8c7e83873d63 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:06.551722 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69f447-3e07-4d7b-a35e-8c7e83873d63 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:06.552027 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69f447-3e07-4d7b-a35e-8c7e83873d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:06.581187 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-bd69f447-3e07-4d7b-a35e-8c7e83873d63 None None] MFA Rules not processed for user `7e0da36180ec477da9789cf6acebc58c`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:56:06.582436 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-bd69f447-3e07-4d7b-a35e-8c7e83873d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:06.652756 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1299/2629] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:56:06 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:56:06.664372 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d6d2fd-18d9-4184-9363-73649ea872bd admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:06.667253 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-26d6d2fd-18d9-4184-9363-73649ea872bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:06.682654 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:56:06.726396 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d6d2fd-18d9-4184-9363-73649ea872bd admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:06.731102 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a842bc-78c7-4979-b1d3-171cb4fe058c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:06.731714 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a842bc-78c7-4979-b1d3-171cb4fe058c service cinder] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:06.731996 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a842bc-78c7-4979-b1d3-171cb4fe058c service cinder] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:06.732272 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a842bc-78c7-4979-b1d3-171cb4fe058c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:06.741506 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a842bc-78c7-4979-b1d3-171cb4fe058c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:06.742942 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a842bc-78c7-4979-b1d3-171cb4fe058c service cinder] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:06.752212 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1331/2630] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:56:06 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:56:07.963210 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd69f447-3e07-4d7b-a35e-8c7e83873d63 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:08.027405 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd69f447-3e07-4d7b-a35e-8c7e83873d63 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:08.031975 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb4d695-c0f4-4f26-bbe9-47b85c712c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:08.032604 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb4d695-c0f4-4f26-bbe9-47b85c712c83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:08.032919 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb4d695-c0f4-4f26-bbe9-47b85c712c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:08.033226 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb4d695-c0f4-4f26-bbe9-47b85c712c83 admin admin] PATH_INFO: `/v3/users/30ebdbf5ab3d4cfda9bee871a930dccf` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:08.034360 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb4d695-c0f4-4f26-bbe9-47b85c712c83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30ebdbf5ab3d4cfda9bee871a930dccf)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:08.036281 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb4d695-c0f4-4f26-bbe9-47b85c712c83 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:08.188961 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-8eb4d695-c0f4-4f26-bbe9-47b85c712c83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30ebdbf5ab3d4cfda9bee871a930dccf'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:08.201135 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-8eb4d695-c0f4-4f26-bbe9-47b85c712c83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30ebdbf5ab3d4cfda9bee871a930dccf'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:08.208305 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-8eb4d695-c0f4-4f26-bbe9-47b85c712c83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30ebdbf5ab3d4cfda9bee871a930dccf'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:08.209949 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:56:08.318145 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-8eb4d695-c0f4-4f26-bbe9-47b85c712c83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30ebdbf5ab3d4cfda9bee871a930dccf'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:08.318792 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-8eb4d695-c0f4-4f26-bbe9-47b85c712c83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30ebdbf5ab3d4cfda9bee871a930dccf'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:08.325271 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1300/2631] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:56:07 2022] DELETE /identity/v3/users/30ebdbf5ab3d4cfda9bee871a930dccf => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:56:08.330058 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d434de08-be7f-432e-9b86-386b7454fbeb None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:08.330544 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d434de08-be7f-432e-9b86-386b7454fbeb None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:08.330692 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d434de08-be7f-432e-9b86-386b7454fbeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:08.351106 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-d434de08-be7f-432e-9b86-386b7454fbeb None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:56:08.356026 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-d434de08-be7f-432e-9b86-386b7454fbeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:08.412329 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1332/2632] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:56:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:56:08.422572 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb4d695-c0f4-4f26-bbe9-47b85c712c83 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:08.423985 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-8eb4d695-c0f4-4f26-bbe9-47b85c712c83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:08.489728 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb4d695-c0f4-4f26-bbe9-47b85c712c83 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:08.493358 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1111165a-b567-437c-875f-00927bb2beee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:08.494004 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111165a-b567-437c-875f-00927bb2beee service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:08.494332 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111165a-b567-437c-875f-00927bb2beee service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:08.494612 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111165a-b567-437c-875f-00927bb2beee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:08.503712 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1111165a-b567-437c-875f-00927bb2beee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:08.505243 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1111165a-b567-437c-875f-00927bb2beee service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:08.517514 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1301/2633] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:56:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:56:08.986065 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d434de08-be7f-432e-9b86-386b7454fbeb None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:08.989280 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-d434de08-be7f-432e-9b86-386b7454fbeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:09.004245 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d434de08-be7f-432e-9b86-386b7454fbeb None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:09.009143 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8602210d-d69e-44dd-bf2f-1443d19a9e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:09.010488 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8602210d-d69e-44dd-bf2f-1443d19a9e30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:09.010980 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8602210d-d69e-44dd-bf2f-1443d19a9e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:09.011361 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8602210d-d69e-44dd-bf2f-1443d19a9e30 admin admin] PATH_INFO: `/v3/projects/44142c9c52034f79b1f5faac920c3f49` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:09.012514 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8602210d-d69e-44dd-bf2f-1443d19a9e30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44142c9c52034f79b1f5faac920c3f49)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:09.014873 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8602210d-d69e-44dd-bf2f-1443d19a9e30 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:09.119909 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-8602210d-d69e-44dd-bf2f-1443d19a9e30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44142c9c52034f79b1f5faac920c3f49', 'request_id': 'req-8602210d-d69e-44dd-bf2f-1443d19a9e30'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:09.125438 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-8602210d-d69e-44dd-bf2f-1443d19a9e30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44142c9c52034f79b1f5faac920c3f49', 'request_id': 'req-8602210d-d69e-44dd-bf2f-1443d19a9e30'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:09.131508 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-8602210d-d69e-44dd-bf2f-1443d19a9e30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44142c9c52034f79b1f5faac920c3f49', 'request_id': 'req-8602210d-d69e-44dd-bf2f-1443d19a9e30'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:09.179650 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-8602210d-d69e-44dd-bf2f-1443d19a9e30 admin admin] The token cache is being invalidate because project 44142c9c52034f79b1f5faac920c3f49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:56:09.179943 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-8602210d-d69e-44dd-bf2f-1443d19a9e30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:09.180595 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-8602210d-d69e-44dd-bf2f-1443d19a9e30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:09.183315 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1333/2634] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:56:08 2022] DELETE /identity/v3/projects/44142c9c52034f79b1f5faac920c3f49 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:56:09.187331 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1111165a-b567-437c-875f-00927bb2beee service neutron] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:09.199733 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1111165a-b567-437c-875f-00927bb2beee service neutron] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:09.263183 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6073a6c5-0995-4e81-ab41-e1986ea14469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:09.263821 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6073a6c5-0995-4e81-ab41-e1986ea14469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:09.264104 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6073a6c5-0995-4e81-ab41-e1986ea14469 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:09.264383 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6073a6c5-0995-4e81-ab41-e1986ea14469 admin admin] PATH_INFO: `/v3/users/7e0da36180ec477da9789cf6acebc58c` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:09.265622 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6073a6c5-0995-4e81-ab41-e1986ea14469 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e0da36180ec477da9789cf6acebc58c)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:09.267483 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6073a6c5-0995-4e81-ab41-e1986ea14469 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:09.423783 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-6073a6c5-0995-4e81-ab41-e1986ea14469 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e0da36180ec477da9789cf6acebc58c'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:09.437223 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-6073a6c5-0995-4e81-ab41-e1986ea14469 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e0da36180ec477da9789cf6acebc58c'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:09.444040 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-6073a6c5-0995-4e81-ab41-e1986ea14469 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e0da36180ec477da9789cf6acebc58c'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:09.506200 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-6073a6c5-0995-4e81-ab41-e1986ea14469 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e0da36180ec477da9789cf6acebc58c'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:09.506812 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-6073a6c5-0995-4e81-ab41-e1986ea14469 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e0da36180ec477da9789cf6acebc58c'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:09.513423 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1302/2635] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:56:09 2022] DELETE /identity/v3/users/7e0da36180ec477da9789cf6acebc58c => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:56:09.976281 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8602210d-d69e-44dd-bf2f-1443d19a9e30 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:09.977773 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-8602210d-d69e-44dd-bf2f-1443d19a9e30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:09.992703 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8602210d-d69e-44dd-bf2f-1443d19a9e30 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:10.034705 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:56:10.052547 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c68f6-2081-46cd-a523-c56e55a8d40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:10.053167 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c68f6-2081-46cd-a523-c56e55a8d40c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:10.053452 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c68f6-2081-46cd-a523-c56e55a8d40c admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:10.053728 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c68f6-2081-46cd-a523-c56e55a8d40c admin admin] PATH_INFO: `/v3/projects/85cfdd48f9b242d890157f0a28cb0e40` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:10.054894 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87c68f6-2081-46cd-a523-c56e55a8d40c admin admin] RBAC: Authorizing `identity:delete_project(project_id=85cfdd48f9b242d890157f0a28cb0e40)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:10.056925 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87c68f6-2081-46cd-a523-c56e55a8d40c admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:10.176200 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-f87c68f6-2081-46cd-a523-c56e55a8d40c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85cfdd48f9b242d890157f0a28cb0e40', 'request_id': 'req-f87c68f6-2081-46cd-a523-c56e55a8d40c'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:10.181817 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-f87c68f6-2081-46cd-a523-c56e55a8d40c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85cfdd48f9b242d890157f0a28cb0e40', 'request_id': 'req-f87c68f6-2081-46cd-a523-c56e55a8d40c'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:10.187928 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-f87c68f6-2081-46cd-a523-c56e55a8d40c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85cfdd48f9b242d890157f0a28cb0e40', 'request_id': 'req-f87c68f6-2081-46cd-a523-c56e55a8d40c'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:10.285372 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-f87c68f6-2081-46cd-a523-c56e55a8d40c admin admin] The token cache is being invalidate because project 85cfdd48f9b242d890157f0a28cb0e40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:56:10.285745 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-f87c68f6-2081-46cd-a523-c56e55a8d40c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:10.286515 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-f87c68f6-2081-46cd-a523-c56e55a8d40c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:10.288045 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1334/2636] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:56:09 2022] DELETE /identity/v3/projects/85cfdd48f9b242d890157f0a28cb0e40 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:56:10.293367 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511cc1c4-2d2a-479a-88b2-d57e595da3a4 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:10.293860 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511cc1c4-2d2a-479a-88b2-d57e595da3a4 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:10.297455 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511cc1c4-2d2a-479a-88b2-d57e595da3a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:10.314251 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-511cc1c4-2d2a-479a-88b2-d57e595da3a4 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:56:10.315391 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-511cc1c4-2d2a-479a-88b2-d57e595da3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:10.324357 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:56:10.374414 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1303/2637] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:56:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:56:10.382024 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7771c7cb-be70-4e27-9872-7bb04bcf4d21 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:10.382308 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7771c7cb-be70-4e27-9872-7bb04bcf4d21 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:10.382588 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7771c7cb-be70-4e27-9872-7bb04bcf4d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:10.402936 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-7771c7cb-be70-4e27-9872-7bb04bcf4d21 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:56:10.404146 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-7771c7cb-be70-4e27-9872-7bb04bcf4d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:10.412195 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1335/2638] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:56:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:56:10.417087 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511cc1c4-2d2a-479a-88b2-d57e595da3a4 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:10.427495 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511cc1c4-2d2a-479a-88b2-d57e595da3a4 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:10.438454 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b162a-583e-4abe-a380-170c6678fcc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:10.439069 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b162a-583e-4abe-a380-170c6678fcc3 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:10.439352 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b162a-583e-4abe-a380-170c6678fcc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:10.439626 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b162a-583e-4abe-a380-170c6678fcc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:10.440368 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5b162a-583e-4abe-a380-170c6678fcc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:56:10.440809 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5b162a-583e-4abe-a380-170c6678fcc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:10.442442 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5b162a-583e-4abe-a380-170c6678fcc3 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:10.457860 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1304/2639] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:56:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:56:10.462346 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7771c7cb-be70-4e27-9872-7bb04bcf4d21 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:10.471999 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7771c7cb-be70-4e27-9872-7bb04bcf4d21 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:10.477178 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e897fb3d-9c5a-4db9-a434-4ec9561c7143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:10.477715 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897fb3d-9c5a-4db9-a434-4ec9561c7143 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:10.477990 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897fb3d-9c5a-4db9-a434-4ec9561c7143 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:10.478392 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897fb3d-9c5a-4db9-a434-4ec9561c7143 admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:10.479144 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e897fb3d-9c5a-4db9-a434-4ec9561c7143 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:10.481085 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e897fb3d-9c5a-4db9-a434-4ec9561c7143 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:10.587790 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1336/2640] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:56:10 2022] POST /identity/v3/projects => generated 369 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:56:10.592321 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b162a-583e-4abe-a380-170c6678fcc3 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:10.604343 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b162a-583e-4abe-a380-170c6678fcc3 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:10.667028 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5224ece-05fb-4b50-9ac6-cd7f44daa8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:10.673742 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5224ece-05fb-4b50-9ac6-cd7f44daa8d0 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:10.673742 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5224ece-05fb-4b50-9ac6-cd7f44daa8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:10.673742 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5224ece-05fb-4b50-9ac6-cd7f44daa8d0 admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:10.673742 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5224ece-05fb-4b50-9ac6-cd7f44daa8d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:10.673742 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5224ece-05fb-4b50-9ac6-cd7f44daa8d0 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:10.798404 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1305/2641] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:56:10 2022] POST /identity/v3/users => generated 285 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:56:10.802533 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e897fb3d-9c5a-4db9-a434-4ec9561c7143 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:10.813457 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e897fb3d-9c5a-4db9-a434-4ec9561c7143 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:10.818752 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d7d677-8041-4eb4-a78b-7aa9b0ff0c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:10.819303 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d7d677-8041-4eb4-a78b-7aa9b0ff0c02 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:10.819585 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d7d677-8041-4eb4-a78b-7aa9b0ff0c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:10.819860 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d7d677-8041-4eb4-a78b-7aa9b0ff0c02 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:10.820510 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d7d677-8041-4eb4-a78b-7aa9b0ff0c02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:10.822135 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d7d677-8041-4eb4-a78b-7aa9b0ff0c02 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:10.832738 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1337/2642] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:56:10.836980 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5224ece-05fb-4b50-9ac6-cd7f44daa8d0 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:10.849022 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5224ece-05fb-4b50-9ac6-cd7f44daa8d0 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:10.856664 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88058aac-bafd-4eaa-ad14-33b90a8d2b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:10.857821 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88058aac-bafd-4eaa-ad14-33b90a8d2b51 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:10.858290 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88058aac-bafd-4eaa-ad14-33b90a8d2b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:10.858570 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88058aac-bafd-4eaa-ad14-33b90a8d2b51 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:10.859228 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88058aac-bafd-4eaa-ad14-33b90a8d2b51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:10.860855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88058aac-bafd-4eaa-ad14-33b90a8d2b51 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:10.873658 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1306/2643] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:56:10.878353 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d7d677-8041-4eb4-a78b-7aa9b0ff0c02 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:10.889734 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d7d677-8041-4eb4-a78b-7aa9b0ff0c02 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:10.895277 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1658df6-4276-4c94-9c15-1cd3f126a3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:10.902904 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1658df6-4276-4c94-9c15-1cd3f126a3f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:10.902904 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1658df6-4276-4c94-9c15-1cd3f126a3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:10.902904 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1658df6-4276-4c94-9c15-1cd3f126a3f7 admin admin] PATH_INFO: `/v3/projects/4fa50532ff1941e8bd9fcc6668d79a73/users/0f54fd79e55d4580acf250f3b3c599d3/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:10.919322 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1658df6-4276-4c94-9c15-1cd3f126a3f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fa50532ff1941e8bd9fcc6668d79a73, user_id=0f54fd79e55d4580acf250f3b3c599d3, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:10.928687 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1658df6-4276-4c94-9c15-1cd3f126a3f7 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:11.012342 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1338/2644] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:56:10 2022] PUT /identity/v3/projects/4fa50532ff1941e8bd9fcc6668d79a73/users/0f54fd79e55d4580acf250f3b3c599d3/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:56:11.024334 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e384b2-5936-43a5-bb0f-89af8fe12af2 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:11.024711 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e384b2-5936-43a5-bb0f-89af8fe12af2 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:11.025813 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e384b2-5936-43a5-bb0f-89af8fe12af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:11.058460 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-54e384b2-5936-43a5-bb0f-89af8fe12af2 None None] MFA Rules not processed for user `0f54fd79e55d4580acf250f3b3c599d3`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:56:11.059809 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-54e384b2-5936-43a5-bb0f-89af8fe12af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:11.129141 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1307/2645] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:56:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:56:11.133610 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1658df6-4276-4c94-9c15-1cd3f126a3f7 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:11.196005 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1658df6-4276-4c94-9c15-1cd3f126a3f7 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:11.200990 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f78b2-dedb-4325-b339-05ea55f4c3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:11.203753 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f78b2-dedb-4325-b339-05ea55f4c3d1 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:11.203753 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f78b2-dedb-4325-b339-05ea55f4c3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:11.203753 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f78b2-dedb-4325-b339-05ea55f4c3d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:11.203753 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0f78b2-dedb-4325-b339-05ea55f4c3d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:11.204619 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0f78b2-dedb-4325-b339-05ea55f4c3d1 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:11.217078 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:56:11.353930 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1339/2646] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:56:11 2022] POST /identity/v3/projects => generated 369 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:56:11.365051 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e384b2-5936-43a5-bb0f-89af8fe12af2 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:11.377094 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e384b2-5936-43a5-bb0f-89af8fe12af2 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:11.387481 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:56:11.433653 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c8a653-7afc-429d-91cd-4b25d6f22965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:11.434248 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c8a653-7afc-429d-91cd-4b25d6f22965 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:11.434526 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c8a653-7afc-429d-91cd-4b25d6f22965 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:11.434797 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c8a653-7afc-429d-91cd-4b25d6f22965 admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:11.435428 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c8a653-7afc-429d-91cd-4b25d6f22965 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:11.437374 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c8a653-7afc-429d-91cd-4b25d6f22965 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:11.534898 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1308/2647] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:56:11 2022] POST /identity/v3/users => generated 291 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:56:11.538909 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f78b2-dedb-4325-b339-05ea55f4c3d1 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:11.550536 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f78b2-dedb-4325-b339-05ea55f4c3d1 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:11.555653 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17232518-ec06-4fab-ae16-04c4af86df62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:11.556961 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17232518-ec06-4fab-ae16-04c4af86df62 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:11.560676 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17232518-ec06-4fab-ae16-04c4af86df62 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:11.560676 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17232518-ec06-4fab-ae16-04c4af86df62 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:11.560676 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17232518-ec06-4fab-ae16-04c4af86df62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:11.560676 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17232518-ec06-4fab-ae16-04c4af86df62 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:11.570503 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1340/2648] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:56:11.574726 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c8a653-7afc-429d-91cd-4b25d6f22965 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:11.584820 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c8a653-7afc-429d-91cd-4b25d6f22965 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:11.590263 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23704d9e-fab8-4d34-9e7b-6a8bc107bfb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:11.590931 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23704d9e-fab8-4d34-9e7b-6a8bc107bfb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:11.591204 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23704d9e-fab8-4d34-9e7b-6a8bc107bfb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:11.591505 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23704d9e-fab8-4d34-9e7b-6a8bc107bfb3 admin admin] PATH_INFO: `/v3/projects/25665a0701b74ba29eece6a3dbee4ba4/users/aecf74dd3b9446aeb76972bee159ea93/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:11.607361 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23704d9e-fab8-4d34-9e7b-6a8bc107bfb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25665a0701b74ba29eece6a3dbee4ba4, user_id=aecf74dd3b9446aeb76972bee159ea93, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:11.610455 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23704d9e-fab8-4d34-9e7b-6a8bc107bfb3 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:11.724269 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1309/2649] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:56:11 2022] PUT /identity/v3/projects/25665a0701b74ba29eece6a3dbee4ba4/users/aecf74dd3b9446aeb76972bee159ea93/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:56:11.728497 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17232518-ec06-4fab-ae16-04c4af86df62 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:11.739843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17232518-ec06-4fab-ae16-04c4af86df62 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:11.795698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e6eeee-8582-409a-b9ac-e9c66b0914c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:11.796300 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e6eeee-8582-409a-b9ac-e9c66b0914c3 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:11.796577 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e6eeee-8582-409a-b9ac-e9c66b0914c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:11.796877 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e6eeee-8582-409a-b9ac-e9c66b0914c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:11.797537 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e6eeee-8582-409a-b9ac-e9c66b0914c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:11.799119 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e6eeee-8582-409a-b9ac-e9c66b0914c3 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:11.809450 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1341/2650] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:56:11.813750 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23704d9e-fab8-4d34-9e7b-6a8bc107bfb3 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:11.824073 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23704d9e-fab8-4d34-9e7b-6a8bc107bfb3 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:11.828822 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f6c723-2127-4928-a06f-a961ce029ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:11.829466 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f6c723-2127-4928-a06f-a961ce029ca8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:11.829742 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f6c723-2127-4928-a06f-a961ce029ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:11.830011 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f6c723-2127-4928-a06f-a961ce029ca8 admin admin] PATH_INFO: `/v3/projects/25665a0701b74ba29eece6a3dbee4ba4/users/aecf74dd3b9446aeb76972bee159ea93/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:11.831880 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f6c723-2127-4928-a06f-a961ce029ca8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25665a0701b74ba29eece6a3dbee4ba4, user_id=aecf74dd3b9446aeb76972bee159ea93, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:11.834543 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f6c723-2127-4928-a06f-a961ce029ca8 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:11.845557 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1310/2651] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:56:11 2022] PUT /identity/v3/projects/25665a0701b74ba29eece6a3dbee4ba4/users/aecf74dd3b9446aeb76972bee159ea93/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:56:11.857444 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db9981-c8ec-4988-b2f4-62d68173ac2f None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:11.857749 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db9981-c8ec-4988-b2f4-62d68173ac2f None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:11.858021 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db9981-c8ec-4988-b2f4-62d68173ac2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:11.886218 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-25db9981-c8ec-4988-b2f4-62d68173ac2f None None] MFA Rules not processed for user `aecf74dd3b9446aeb76972bee159ea93`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:56:11.887421 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-25db9981-c8ec-4988-b2f4-62d68173ac2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:11.956563 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1342/2652] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:56:11 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:56:11.967877 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f6c723-2127-4928-a06f-a961ce029ca8 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:11.969684 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-e2f6c723-2127-4928-a06f-a961ce029ca8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:12.028343 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f6c723-2127-4928-a06f-a961ce029ca8 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:12.031935 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a716cc9a-7899-4449-9cb8-dc9887560f5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:12.032541 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a716cc9a-7899-4449-9cb8-dc9887560f5c service cinder] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:12.032861 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a716cc9a-7899-4449-9cb8-dc9887560f5c service cinder] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:12.033136 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a716cc9a-7899-4449-9cb8-dc9887560f5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:12.042639 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a716cc9a-7899-4449-9cb8-dc9887560f5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:12.044070 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a716cc9a-7899-4449-9cb8-dc9887560f5c service cinder] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:12.059490 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1311/2653] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:56:11 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:56:13.687933 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25db9981-c8ec-4988-b2f4-62d68173ac2f None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:13.689610 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-25db9981-c8ec-4988-b2f4-62d68173ac2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:13.758473 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:56:13.779857 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25db9981-c8ec-4988-b2f4-62d68173ac2f None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:13.783838 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a26f7-6f6f-490d-83cb-7e6a127357f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:13.785474 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a26f7-6f6f-490d-83cb-7e6a127357f7 service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:13.785474 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a26f7-6f6f-490d-83cb-7e6a127357f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:13.785474 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a26f7-6f6f-490d-83cb-7e6a127357f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:13.786779 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-dd9a26f7-6f6f-490d-83cb-7e6a127357f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:13.872279 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9a26f7-6f6f-490d-83cb-7e6a127357f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:13.873899 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9a26f7-6f6f-490d-83cb-7e6a127357f7 service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:13.883663 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1343/2654] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:56:13 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:56:14.944188 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a716cc9a-7899-4449-9cb8-dc9887560f5c service cinder] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:15.017967 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:56:15.069316 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a716cc9a-7899-4449-9cb8-dc9887560f5c service cinder] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:15.074581 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab13111-b693-447e-a98f-13962bb57598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:15.075239 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab13111-b693-447e-a98f-13962bb57598 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:15.075516 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab13111-b693-447e-a98f-13962bb57598 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:15.075842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab13111-b693-447e-a98f-13962bb57598 admin admin] PATH_INFO: `/v3/users/0f54fd79e55d4580acf250f3b3c599d3` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:15.077116 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab13111-b693-447e-a98f-13962bb57598 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f54fd79e55d4580acf250f3b3c599d3)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:15.079057 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab13111-b693-447e-a98f-13962bb57598 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:15.254945 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-7ab13111-b693-447e-a98f-13962bb57598 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f54fd79e55d4580acf250f3b3c599d3'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:15.272170 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-7ab13111-b693-447e-a98f-13962bb57598 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f54fd79e55d4580acf250f3b3c599d3'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:15.281394 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-7ab13111-b693-447e-a98f-13962bb57598 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f54fd79e55d4580acf250f3b3c599d3'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:15.346427 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-7ab13111-b693-447e-a98f-13962bb57598 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f54fd79e55d4580acf250f3b3c599d3'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:15.347102 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-7ab13111-b693-447e-a98f-13962bb57598 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f54fd79e55d4580acf250f3b3c599d3'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:15.355453 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1312/2655] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:56:14 2022] DELETE /identity/v3/users/0f54fd79e55d4580acf250f3b3c599d3 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:56:15.360260 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a8b6d-936b-46b9-a767-74e8f0b14163 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:15.376469 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a8b6d-936b-46b9-a767-74e8f0b14163 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:15.376469 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a8b6d-936b-46b9-a767-74e8f0b14163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:15.402766 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-619a8b6d-936b-46b9-a767-74e8f0b14163 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:56:15.404017 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-619a8b6d-936b-46b9-a767-74e8f0b14163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:15.474313 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1344/2656] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:56:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:56:15.484423 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab13111-b693-447e-a98f-13962bb57598 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:15.487288 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-7ab13111-b693-447e-a98f-13962bb57598 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:15.569573 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab13111-b693-447e-a98f-13962bb57598 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:15.577805 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d36b0a-15fc-4828-a25a-b90585368d02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:15.578747 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d36b0a-15fc-4828-a25a-b90585368d02 service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:15.578747 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d36b0a-15fc-4828-a25a-b90585368d02 service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:15.579076 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d36b0a-15fc-4828-a25a-b90585368d02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:15.589820 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d36b0a-15fc-4828-a25a-b90585368d02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:15.591256 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d36b0a-15fc-4828-a25a-b90585368d02 service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:15.602421 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1313/2657] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:56:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:56:16.018836 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a8b6d-936b-46b9-a767-74e8f0b14163 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:16.021113 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-619a8b6d-936b-46b9-a767-74e8f0b14163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:16.038525 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a8b6d-936b-46b9-a767-74e8f0b14163 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:16.043650 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfeb174-6d6e-40c9-905f-9f6ff61d47c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:16.046620 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfeb174-6d6e-40c9-905f-9f6ff61d47c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:16.046620 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfeb174-6d6e-40c9-905f-9f6ff61d47c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:16.046620 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfeb174-6d6e-40c9-905f-9f6ff61d47c1 admin admin] PATH_INFO: `/v3/projects/4fa50532ff1941e8bd9fcc6668d79a73` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:16.046620 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfeb174-6d6e-40c9-905f-9f6ff61d47c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fa50532ff1941e8bd9fcc6668d79a73)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:16.047914 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfeb174-6d6e-40c9-905f-9f6ff61d47c1 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:16.154476 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:56:16.160963 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-4cfeb174-6d6e-40c9-905f-9f6ff61d47c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fa50532ff1941e8bd9fcc6668d79a73', 'request_id': 'req-4cfeb174-6d6e-40c9-905f-9f6ff61d47c1'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:16.168174 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-4cfeb174-6d6e-40c9-905f-9f6ff61d47c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fa50532ff1941e8bd9fcc6668d79a73', 'request_id': 'req-4cfeb174-6d6e-40c9-905f-9f6ff61d47c1'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:16.175653 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-4cfeb174-6d6e-40c9-905f-9f6ff61d47c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fa50532ff1941e8bd9fcc6668d79a73', 'request_id': 'req-4cfeb174-6d6e-40c9-905f-9f6ff61d47c1'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:16.267409 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-4cfeb174-6d6e-40c9-905f-9f6ff61d47c1 admin admin] The token cache is being invalidate because project 4fa50532ff1941e8bd9fcc6668d79a73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:56:16.267712 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-4cfeb174-6d6e-40c9-905f-9f6ff61d47c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:16.268437 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-4cfeb174-6d6e-40c9-905f-9f6ff61d47c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:16.271727 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1345/2658] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:56:16 2022] DELETE /identity/v3/projects/4fa50532ff1941e8bd9fcc6668d79a73 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:56:16.276094 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d36b0a-15fc-4828-a25a-b90585368d02 service neutron] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:16.286337 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d36b0a-15fc-4828-a25a-b90585368d02 service neutron] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:16.332397 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:56:16.348519 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9521948a-4358-4fd2-a537-73ba76ffcefc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:16.349329 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521948a-4358-4fd2-a537-73ba76ffcefc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:16.349642 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521948a-4358-4fd2-a537-73ba76ffcefc admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:16.349915 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521948a-4358-4fd2-a537-73ba76ffcefc admin admin] PATH_INFO: `/v3/users/aecf74dd3b9446aeb76972bee159ea93` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:16.351127 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9521948a-4358-4fd2-a537-73ba76ffcefc admin admin] RBAC: Authorizing `identity:delete_user(user_id=aecf74dd3b9446aeb76972bee159ea93)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:16.353774 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9521948a-4358-4fd2-a537-73ba76ffcefc admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:16.525140 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-9521948a-4358-4fd2-a537-73ba76ffcefc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aecf74dd3b9446aeb76972bee159ea93'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:16.554513 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-9521948a-4358-4fd2-a537-73ba76ffcefc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aecf74dd3b9446aeb76972bee159ea93'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:16.565651 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-9521948a-4358-4fd2-a537-73ba76ffcefc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aecf74dd3b9446aeb76972bee159ea93'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:16.634269 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-9521948a-4358-4fd2-a537-73ba76ffcefc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aecf74dd3b9446aeb76972bee159ea93'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:16.635523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-9521948a-4358-4fd2-a537-73ba76ffcefc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aecf74dd3b9446aeb76972bee159ea93'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:16.644080 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1314/2659] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:56:16 2022] DELETE /identity/v3/users/aecf74dd3b9446aeb76972bee159ea93 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:56:17.147802 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfeb174-6d6e-40c9-905f-9f6ff61d47c1 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:17.150647 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-4cfeb174-6d6e-40c9-905f-9f6ff61d47c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:17.165815 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfeb174-6d6e-40c9-905f-9f6ff61d47c1 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:17.229619 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce05788-29ce-40da-a47b-4c66335df01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:17.230538 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce05788-29ce-40da-a47b-4c66335df01c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:17.230817 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce05788-29ce-40da-a47b-4c66335df01c admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:17.231085 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce05788-29ce-40da-a47b-4c66335df01c admin admin] PATH_INFO: `/v3/projects/25665a0701b74ba29eece6a3dbee4ba4` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:17.232247 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce05788-29ce-40da-a47b-4c66335df01c admin admin] RBAC: Authorizing `identity:delete_project(project_id=25665a0701b74ba29eece6a3dbee4ba4)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:17.234215 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce05788-29ce-40da-a47b-4c66335df01c admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:17.335152 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-6ce05788-29ce-40da-a47b-4c66335df01c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25665a0701b74ba29eece6a3dbee4ba4', 'request_id': 'req-6ce05788-29ce-40da-a47b-4c66335df01c'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:17.345256 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-6ce05788-29ce-40da-a47b-4c66335df01c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25665a0701b74ba29eece6a3dbee4ba4', 'request_id': 'req-6ce05788-29ce-40da-a47b-4c66335df01c'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:17.350819 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-6ce05788-29ce-40da-a47b-4c66335df01c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25665a0701b74ba29eece6a3dbee4ba4', 'request_id': 'req-6ce05788-29ce-40da-a47b-4c66335df01c'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:17.456137 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-6ce05788-29ce-40da-a47b-4c66335df01c admin admin] The token cache is being invalidate because project 25665a0701b74ba29eece6a3dbee4ba4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:56:17.456471 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-6ce05788-29ce-40da-a47b-4c66335df01c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:17.457485 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-6ce05788-29ce-40da-a47b-4c66335df01c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:17.458774 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1346/2660] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:56:17 2022] DELETE /identity/v3/projects/25665a0701b74ba29eece6a3dbee4ba4 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:56:17.465233 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e64cdab-3801-46a2-baf6-d61bf37024be None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:17.465738 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e64cdab-3801-46a2-baf6-d61bf37024be None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:17.465834 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e64cdab-3801-46a2-baf6-d61bf37024be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:17.517275 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-5e64cdab-3801-46a2-baf6-d61bf37024be None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:56:17.518479 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-5e64cdab-3801-46a2-baf6-d61bf37024be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:17.582099 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1315/2661] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:56:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:56:17.590660 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45487eb6-b883-4127-bf6a-904125869338 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:17.590949 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45487eb6-b883-4127-bf6a-904125869338 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:17.591224 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45487eb6-b883-4127-bf6a-904125869338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:17.612567 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-45487eb6-b883-4127-bf6a-904125869338 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:56:17.613974 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-45487eb6-b883-4127-bf6a-904125869338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:17.622772 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1347/2662] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:56:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:56:17.627315 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e64cdab-3801-46a2-baf6-d61bf37024be None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:17.638679 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e64cdab-3801-46a2-baf6-d61bf37024be None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:17.643546 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b320c4f-4202-4205-a859-201109659c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:17.644156 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b320c4f-4202-4205-a859-201109659c9d admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:17.644436 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b320c4f-4202-4205-a859-201109659c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:17.644698 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b320c4f-4202-4205-a859-201109659c9d admin admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:17.645385 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b320c4f-4202-4205-a859-201109659c9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:56:17.645690 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b320c4f-4202-4205-a859-201109659c9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:17.647299 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b320c4f-4202-4205-a859-201109659c9d admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:17.660295 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1316/2663] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:56:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:56:17.666380 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45487eb6-b883-4127-bf6a-904125869338 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:17.676610 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45487eb6-b883-4127-bf6a-904125869338 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:17.682150 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96798bb5-6aed-4de2-91cc-75ed90d21ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:17.682743 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96798bb5-6aed-4de2-91cc-75ed90d21ab2 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:17.683017 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96798bb5-6aed-4de2-91cc-75ed90d21ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:17.683283 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96798bb5-6aed-4de2-91cc-75ed90d21ab2 admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:17.683977 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96798bb5-6aed-4de2-91cc-75ed90d21ab2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:17.685906 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96798bb5-6aed-4de2-91cc-75ed90d21ab2 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:17.787616 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1348/2664] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:56:17 2022] POST /identity/v3/projects => generated 401 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:56:17.792596 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b320c4f-4202-4205-a859-201109659c9d admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:17.802634 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b320c4f-4202-4205-a859-201109659c9d admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:17.828245 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:56:17.861270 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e8fee-2825-47df-a40b-33e67f9e3048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:17.861876 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e8fee-2825-47df-a40b-33e67f9e3048 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:17.862153 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e8fee-2825-47df-a40b-33e67f9e3048 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:17.862425 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e8fee-2825-47df-a40b-33e67f9e3048 admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:17.863044 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360e8fee-2825-47df-a40b-33e67f9e3048 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:17.865027 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360e8fee-2825-47df-a40b-33e67f9e3048 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:17.951103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1317/2665] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:56:17 2022] POST /identity/v3/users => generated 301 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:56:17.955465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96798bb5-6aed-4de2-91cc-75ed90d21ab2 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:17.966583 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96798bb5-6aed-4de2-91cc-75ed90d21ab2 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:17.971333 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11facf6b-e2d0-4f8b-ad34-3535c79add9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:17.971946 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11facf6b-e2d0-4f8b-ad34-3535c79add9d admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:17.972210 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11facf6b-e2d0-4f8b-ad34-3535c79add9d admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:17.972484 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11facf6b-e2d0-4f8b-ad34-3535c79add9d admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:17.973164 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11facf6b-e2d0-4f8b-ad34-3535c79add9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:17.974745 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11facf6b-e2d0-4f8b-ad34-3535c79add9d admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:17.989144 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1349/2666] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:56:17.996374 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e8fee-2825-47df-a40b-33e67f9e3048 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:18.006608 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e8fee-2825-47df-a40b-33e67f9e3048 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:18.011341 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ab228-8042-48f1-a730-7907c29e2905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:18.011926 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ab228-8042-48f1-a730-7907c29e2905 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:18.012203 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ab228-8042-48f1-a730-7907c29e2905 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:18.012475 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ab228-8042-48f1-a730-7907c29e2905 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:18.013157 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35ab228-8042-48f1-a730-7907c29e2905 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:18.014732 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35ab228-8042-48f1-a730-7907c29e2905 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:18.025257 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1318/2667] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:56:18.029729 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11facf6b-e2d0-4f8b-ad34-3535c79add9d admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:18.039667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11facf6b-e2d0-4f8b-ad34-3535c79add9d admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:18.045003 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da90a87-19d9-46f9-bae1-65a8f14d7792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:18.045643 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da90a87-19d9-46f9-bae1-65a8f14d7792 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:18.045919 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da90a87-19d9-46f9-bae1-65a8f14d7792 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:18.046191 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da90a87-19d9-46f9-bae1-65a8f14d7792 admin admin] PATH_INFO: `/v3/projects/9a0e23324c4649398f4b9f1eadb0192c/users/d25239bb234f46c6869cc5ffbd56f33c/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:18.062256 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da90a87-19d9-46f9-bae1-65a8f14d7792 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a0e23324c4649398f4b9f1eadb0192c, user_id=d25239bb234f46c6869cc5ffbd56f33c, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:18.064926 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da90a87-19d9-46f9-bae1-65a8f14d7792 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:18.067999 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:56:18.145991 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1350/2668] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:56:18 2022] PUT /identity/v3/projects/9a0e23324c4649398f4b9f1eadb0192c/users/d25239bb234f46c6869cc5ffbd56f33c/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:56:18.158410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f109f7-50e8-498b-a3d1-a2349523db6c None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:18.158694 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f109f7-50e8-498b-a3d1-a2349523db6c None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:18.158981 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f109f7-50e8-498b-a3d1-a2349523db6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:18.189544 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-f6f109f7-50e8-498b-a3d1-a2349523db6c None None] MFA Rules not processed for user `d25239bb234f46c6869cc5ffbd56f33c`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:56:18.190904 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-f6f109f7-50e8-498b-a3d1-a2349523db6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:18.269894 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1319/2669] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:56:18 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:56:18.274381 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da90a87-19d9-46f9-bae1-65a8f14d7792 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:18.340320 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da90a87-19d9-46f9-bae1-65a8f14d7792 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:18.345043 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43a7c5-3468-483c-88e5-fef2b7f1f859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:18.345647 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43a7c5-3468-483c-88e5-fef2b7f1f859 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:18.345931 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43a7c5-3468-483c-88e5-fef2b7f1f859 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:18.346210 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43a7c5-3468-483c-88e5-fef2b7f1f859 admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:18.346855 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b43a7c5-3468-483c-88e5-fef2b7f1f859 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:18.348777 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b43a7c5-3468-483c-88e5-fef2b7f1f859 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:18.486568 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1351/2670] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:56:18 2022] POST /identity/v3/projects => generated 401 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:56:18.490503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f109f7-50e8-498b-a3d1-a2349523db6c None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:18.501953 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f109f7-50e8-498b-a3d1-a2349523db6c None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:18.561136 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff46d80c-978c-49e9-9c9a-ca85be92f9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:18.561769 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff46d80c-978c-49e9-9c9a-ca85be92f9f7 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:18.562040 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff46d80c-978c-49e9-9c9a-ca85be92f9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:18.562308 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff46d80c-978c-49e9-9c9a-ca85be92f9f7 admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:18.562942 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff46d80c-978c-49e9-9c9a-ca85be92f9f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:18.564883 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff46d80c-978c-49e9-9c9a-ca85be92f9f7 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:18.662270 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1320/2671] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:56:18 2022] POST /identity/v3/users => generated 307 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:56:18.666258 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43a7c5-3468-483c-88e5-fef2b7f1f859 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:18.676769 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43a7c5-3468-483c-88e5-fef2b7f1f859 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:18.681683 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16e839d-655c-4ada-b63a-7548078843e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:18.687023 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e839d-655c-4ada-b63a-7548078843e8 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:18.687023 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e839d-655c-4ada-b63a-7548078843e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:18.687023 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e839d-655c-4ada-b63a-7548078843e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:18.687023 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16e839d-655c-4ada-b63a-7548078843e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:18.687023 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16e839d-655c-4ada-b63a-7548078843e8 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:18.696960 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1352/2672] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:56:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:56:18.701681 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff46d80c-978c-49e9-9c9a-ca85be92f9f7 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:18.712101 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff46d80c-978c-49e9-9c9a-ca85be92f9f7 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:18.717834 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffda3e5a-18f4-49de-8d98-22d952ce4371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:18.718503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda3e5a-18f4-49de-8d98-22d952ce4371 admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:18.718785 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda3e5a-18f4-49de-8d98-22d952ce4371 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:18.719061 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda3e5a-18f4-49de-8d98-22d952ce4371 admin admin] PATH_INFO: `/v3/projects/b4275ecf04c24d9abfb4652189f55967/users/ead1bcd99db540fdb06fe26fce3a5d77/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:18.734671 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffda3e5a-18f4-49de-8d98-22d952ce4371 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4275ecf04c24d9abfb4652189f55967, user_id=ead1bcd99db540fdb06fe26fce3a5d77, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:18.737485 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffda3e5a-18f4-49de-8d98-22d952ce4371 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:18.821414 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:56:18.826929 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1321/2673] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:56:18 2022] PUT /identity/v3/projects/b4275ecf04c24d9abfb4652189f55967/users/ead1bcd99db540fdb06fe26fce3a5d77/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:56:18.830923 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16e839d-655c-4ada-b63a-7548078843e8 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:18.841842 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16e839d-655c-4ada-b63a-7548078843e8 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:18.898614 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7400af2-3cfe-4035-b6e9-b1af7145e92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:18.899255 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7400af2-3cfe-4035-b6e9-b1af7145e92d admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:18.899563 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7400af2-3cfe-4035-b6e9-b1af7145e92d admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:18.899831 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7400af2-3cfe-4035-b6e9-b1af7145e92d admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:18.900465 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7400af2-3cfe-4035-b6e9-b1af7145e92d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:18.902296 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7400af2-3cfe-4035-b6e9-b1af7145e92d admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:18.913317 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1353/2674] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:56:18 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:56:18.916853 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffda3e5a-18f4-49de-8d98-22d952ce4371 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:18.927259 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffda3e5a-18f4-49de-8d98-22d952ce4371 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:18.933574 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49196f36-921b-4e9b-b8ec-89958a23fc4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:18.934308 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49196f36-921b-4e9b-b8ec-89958a23fc4e admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:18.934527 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49196f36-921b-4e9b-b8ec-89958a23fc4e admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:18.934803 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49196f36-921b-4e9b-b8ec-89958a23fc4e admin admin] PATH_INFO: `/v3/projects/b4275ecf04c24d9abfb4652189f55967/users/ead1bcd99db540fdb06fe26fce3a5d77/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:18.936748 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49196f36-921b-4e9b-b8ec-89958a23fc4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4275ecf04c24d9abfb4652189f55967, user_id=ead1bcd99db540fdb06fe26fce3a5d77, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:18.939402 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49196f36-921b-4e9b-b8ec-89958a23fc4e admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:18.950880 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1322/2675] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:56:18 2022] PUT /identity/v3/projects/b4275ecf04c24d9abfb4652189f55967/users/ead1bcd99db540fdb06fe26fce3a5d77/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:56:18.961188 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd91eb4-5837-4394-9759-4645404b5bc4 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:18.961471 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd91eb4-5837-4394-9759-4645404b5bc4 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:18.961743 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd91eb4-5837-4394-9759-4645404b5bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:18.990347 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-bbd91eb4-5837-4394-9759-4645404b5bc4 None None] MFA Rules not processed for user `ead1bcd99db540fdb06fe26fce3a5d77`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:56:18.991545 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-bbd91eb4-5837-4394-9759-4645404b5bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:19.042244 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:56:19.059646 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1354/2676] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:56:18 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:56:19.070411 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49196f36-921b-4e9b-b8ec-89958a23fc4e admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:19.072147 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-49196f36-921b-4e9b-b8ec-89958a23fc4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:19.133299 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49196f36-921b-4e9b-b8ec-89958a23fc4e admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:19.137675 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32835d1-6845-449d-beef-443ddd590f21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:19.138273 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32835d1-6845-449d-beef-443ddd590f21 service cinder] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:19.138557 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32835d1-6845-449d-beef-443ddd590f21 service cinder] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:19.138833 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32835d1-6845-449d-beef-443ddd590f21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:19.148129 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32835d1-6845-449d-beef-443ddd590f21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:19.149599 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32835d1-6845-449d-beef-443ddd590f21 service cinder] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:19.159505 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1323/2677] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:56:19 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:56:19.389424 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd91eb4-5837-4394-9759-4645404b5bc4 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:19.399596 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd91eb4-5837-4394-9759-4645404b5bc4 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:19.404099 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc70ae1a-395e-4bed-a2d4-02f577c0de38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:19.404763 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70ae1a-395e-4bed-a2d4-02f577c0de38 service cinder] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:19.405051 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70ae1a-395e-4bed-a2d4-02f577c0de38 service cinder] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:19.405328 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70ae1a-395e-4bed-a2d4-02f577c0de38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:19.465394 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc70ae1a-395e-4bed-a2d4-02f577c0de38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:19.468804 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc70ae1a-395e-4bed-a2d4-02f577c0de38 service cinder] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:19.484759 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1355/2678] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:56:19 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:56:24.550742 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32835d1-6845-449d-beef-443ddd590f21 service cinder] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:24.615785 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32835d1-6845-449d-beef-443ddd590f21 service cinder] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:24.621011 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fcc590-82fc-4f77-bdf6-1c3606686a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:24.621635 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fcc590-82fc-4f77-bdf6-1c3606686a52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:24.621940 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fcc590-82fc-4f77-bdf6-1c3606686a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:24.622213 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fcc590-82fc-4f77-bdf6-1c3606686a52 admin admin] PATH_INFO: `/v3/users/d25239bb234f46c6869cc5ffbd56f33c` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:24.623311 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fcc590-82fc-4f77-bdf6-1c3606686a52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d25239bb234f46c6869cc5ffbd56f33c)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:24.625383 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fcc590-82fc-4f77-bdf6-1c3606686a52 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:24.872461 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-81fcc590-82fc-4f77-bdf6-1c3606686a52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd25239bb234f46c6869cc5ffbd56f33c'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:24.886784 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-81fcc590-82fc-4f77-bdf6-1c3606686a52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd25239bb234f46c6869cc5ffbd56f33c'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:24.894222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-81fcc590-82fc-4f77-bdf6-1c3606686a52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd25239bb234f46c6869cc5ffbd56f33c'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:24.950812 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-81fcc590-82fc-4f77-bdf6-1c3606686a52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd25239bb234f46c6869cc5ffbd56f33c'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:24.951613 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-81fcc590-82fc-4f77-bdf6-1c3606686a52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd25239bb234f46c6869cc5ffbd56f33c'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:24.957999 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1324/2679] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:56:24 2022] DELETE /identity/v3/users/d25239bb234f46c6869cc5ffbd56f33c => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:56:24.963354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258ad19-2119-495b-a397-7464242901ca None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:24.963849 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258ad19-2119-495b-a397-7464242901ca None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:24.963976 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258ad19-2119-495b-a397-7464242901ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:24.983662 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-b258ad19-2119-495b-a397-7464242901ca None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:56:24.984892 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-b258ad19-2119-495b-a397-7464242901ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:25.044512 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1356/2680] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:56:25.054306 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fcc590-82fc-4f77-bdf6-1c3606686a52 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:25.055710 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-81fcc590-82fc-4f77-bdf6-1c3606686a52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:25.067130 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:56:25.115321 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fcc590-82fc-4f77-bdf6-1c3606686a52 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:25.119568 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078795e9-348f-4bae-9bd3-d5fb574a773c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:25.120170 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078795e9-348f-4bae-9bd3-d5fb574a773c service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:25.120452 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078795e9-348f-4bae-9bd3-d5fb574a773c service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:25.120838 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078795e9-348f-4bae-9bd3-d5fb574a773c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:25.130861 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078795e9-348f-4bae-9bd3-d5fb574a773c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:25.132231 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078795e9-348f-4bae-9bd3-d5fb574a773c service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:25.142154 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1325/2681] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:56:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:56:25.548927 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b258ad19-2119-495b-a397-7464242901ca None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:25.553965 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-b258ad19-2119-495b-a397-7464242901ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:25.573293 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b258ad19-2119-495b-a397-7464242901ca None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:25.579829 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb07a2d3-0b75-41ff-a5f0-ab6c4dedfba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:25.580470 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb07a2d3-0b75-41ff-a5f0-ab6c4dedfba5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:25.580777 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb07a2d3-0b75-41ff-a5f0-ab6c4dedfba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:25.582758 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb07a2d3-0b75-41ff-a5f0-ab6c4dedfba5 admin admin] PATH_INFO: `/v3/projects/9a0e23324c4649398f4b9f1eadb0192c` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:25.582758 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb07a2d3-0b75-41ff-a5f0-ab6c4dedfba5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a0e23324c4649398f4b9f1eadb0192c)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:25.584157 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb07a2d3-0b75-41ff-a5f0-ab6c4dedfba5 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:25.786688 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-eb07a2d3-0b75-41ff-a5f0-ab6c4dedfba5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a0e23324c4649398f4b9f1eadb0192c', 'request_id': 'req-eb07a2d3-0b75-41ff-a5f0-ab6c4dedfba5'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:25.792529 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-eb07a2d3-0b75-41ff-a5f0-ab6c4dedfba5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a0e23324c4649398f4b9f1eadb0192c', 'request_id': 'req-eb07a2d3-0b75-41ff-a5f0-ab6c4dedfba5'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:25.798699 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-eb07a2d3-0b75-41ff-a5f0-ab6c4dedfba5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a0e23324c4649398f4b9f1eadb0192c', 'request_id': 'req-eb07a2d3-0b75-41ff-a5f0-ab6c4dedfba5'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:25.870532 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-eb07a2d3-0b75-41ff-a5f0-ab6c4dedfba5 admin admin] The token cache is being invalidate because project 9a0e23324c4649398f4b9f1eadb0192c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:56:25.870821 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-eb07a2d3-0b75-41ff-a5f0-ab6c4dedfba5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:25.871436 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-eb07a2d3-0b75-41ff-a5f0-ab6c4dedfba5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:25.872933 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1357/2682] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:56:25 2022] DELETE /identity/v3/projects/9a0e23324c4649398f4b9f1eadb0192c => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:56:25.877007 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078795e9-348f-4bae-9bd3-d5fb574a773c service neutron] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:25.887136 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078795e9-348f-4bae-9bd3-d5fb574a773c service neutron] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:25.942866 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9476d95f-ad6b-40f2-be2e-858a6c637bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:25.943491 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476d95f-ad6b-40f2-be2e-858a6c637bb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:25.943771 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476d95f-ad6b-40f2-be2e-858a6c637bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:25.944049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476d95f-ad6b-40f2-be2e-858a6c637bb0 admin admin] PATH_INFO: `/v3/users/ead1bcd99db540fdb06fe26fce3a5d77` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:25.945181 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9476d95f-ad6b-40f2-be2e-858a6c637bb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ead1bcd99db540fdb06fe26fce3a5d77)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:25.947801 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9476d95f-ad6b-40f2-be2e-858a6c637bb0 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:26.188695 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-9476d95f-ad6b-40f2-be2e-858a6c637bb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ead1bcd99db540fdb06fe26fce3a5d77'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:26.201185 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-9476d95f-ad6b-40f2-be2e-858a6c637bb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ead1bcd99db540fdb06fe26fce3a5d77'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:26.208241 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-9476d95f-ad6b-40f2-be2e-858a6c637bb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ead1bcd99db540fdb06fe26fce3a5d77'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:26.270931 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-9476d95f-ad6b-40f2-be2e-858a6c637bb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ead1bcd99db540fdb06fe26fce3a5d77'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:26.271593 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-9476d95f-ad6b-40f2-be2e-858a6c637bb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ead1bcd99db540fdb06fe26fce3a5d77'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:26.279909 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1326/2683] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:56:25 2022] DELETE /identity/v3/users/ead1bcd99db540fdb06fe26fce3a5d77 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:56:26.695679 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb07a2d3-0b75-41ff-a5f0-ab6c4dedfba5 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:26.697736 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-eb07a2d3-0b75-41ff-a5f0-ab6c4dedfba5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:26.712362 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb07a2d3-0b75-41ff-a5f0-ab6c4dedfba5 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:26.724509 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:56:26.775680 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2ff0f0-889b-406b-b782-07169c4c6195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:26.776842 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ff0f0-889b-406b-b782-07169c4c6195 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:26.777120 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ff0f0-889b-406b-b782-07169c4c6195 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:26.777390 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ff0f0-889b-406b-b782-07169c4c6195 admin admin] PATH_INFO: `/v3/projects/b4275ecf04c24d9abfb4652189f55967` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:26.778517 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2ff0f0-889b-406b-b782-07169c4c6195 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4275ecf04c24d9abfb4652189f55967)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:26.780464 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2ff0f0-889b-406b-b782-07169c4c6195 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:26.922822 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-0f2ff0f0-889b-406b-b782-07169c4c6195 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4275ecf04c24d9abfb4652189f55967', 'request_id': 'req-0f2ff0f0-889b-406b-b782-07169c4c6195'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:26.928132 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-0f2ff0f0-889b-406b-b782-07169c4c6195 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4275ecf04c24d9abfb4652189f55967', 'request_id': 'req-0f2ff0f0-889b-406b-b782-07169c4c6195'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:26.935828 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-0f2ff0f0-889b-406b-b782-07169c4c6195 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4275ecf04c24d9abfb4652189f55967', 'request_id': 'req-0f2ff0f0-889b-406b-b782-07169c4c6195'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:27.077299 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-0f2ff0f0-889b-406b-b782-07169c4c6195 admin admin] The token cache is being invalidate because project b4275ecf04c24d9abfb4652189f55967 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:56:27.077598 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-0f2ff0f0-889b-406b-b782-07169c4c6195 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:27.078245 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-0f2ff0f0-889b-406b-b782-07169c4c6195 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:27.080165 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1358/2684] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:56:26 2022] DELETE /identity/v3/projects/b4275ecf04c24d9abfb4652189f55967 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:56:27.085481 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c4925-5f36-4714-bd51-ea70db0513ff None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:27.086257 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c4925-5f36-4714-bd51-ea70db0513ff None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:27.086808 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c4925-5f36-4714-bd51-ea70db0513ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:27.109713 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-8d4c4925-5f36-4714-bd51-ea70db0513ff None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:56:27.110967 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-8d4c4925-5f36-4714-bd51-ea70db0513ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:27.139220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:56:27.173131 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1327/2685] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:56:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:56:27.180819 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27825337-1670-485f-850b-1f95077532a8 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:27.181112 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27825337-1670-485f-850b-1f95077532a8 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:27.182127 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27825337-1670-485f-850b-1f95077532a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:27.204489 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-27825337-1670-485f-850b-1f95077532a8 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:56:27.205854 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-27825337-1670-485f-850b-1f95077532a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:27.214614 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1359/2686] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:56:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:56:27.218762 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4c4925-5f36-4714-bd51-ea70db0513ff None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:27.228901 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4c4925-5f36-4714-bd51-ea70db0513ff None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:27.235460 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594e905-158a-469d-8983-cdbd32b747a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:27.235460 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594e905-158a-469d-8983-cdbd32b747a1 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:27.235460 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594e905-158a-469d-8983-cdbd32b747a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:27.235460 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594e905-158a-469d-8983-cdbd32b747a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:27.235845 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f594e905-158a-469d-8983-cdbd32b747a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:56:27.235845 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f594e905-158a-469d-8983-cdbd32b747a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:27.237514 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f594e905-158a-469d-8983-cdbd32b747a1 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:27.250197 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1328/2687] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:56:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:56:27.254746 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27825337-1670-485f-850b-1f95077532a8 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:27.264367 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27825337-1670-485f-850b-1f95077532a8 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:27.269018 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ccae0-8716-4ac7-bbd0-299f1e8ffb69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:27.269606 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ccae0-8716-4ac7-bbd0-299f1e8ffb69 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:27.269886 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ccae0-8716-4ac7-bbd0-299f1e8ffb69 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:27.270158 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ccae0-8716-4ac7-bbd0-299f1e8ffb69 admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:27.270788 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0ccae0-8716-4ac7-bbd0-299f1e8ffb69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:27.272683 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0ccae0-8716-4ac7-bbd0-299f1e8ffb69 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:27.355125 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1360/2688] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:56:27 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:56:27.358112 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594e905-158a-469d-8983-cdbd32b747a1 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:27.371945 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594e905-158a-469d-8983-cdbd32b747a1 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:27.440850 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7dcf6b-44cc-432e-a4ad-051fbb35ec23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:27.441492 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7dcf6b-44cc-432e-a4ad-051fbb35ec23 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:27.441774 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7dcf6b-44cc-432e-a4ad-051fbb35ec23 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:27.442065 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7dcf6b-44cc-432e-a4ad-051fbb35ec23 admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:27.442699 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7dcf6b-44cc-432e-a4ad-051fbb35ec23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:27.444620 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7dcf6b-44cc-432e-a4ad-051fbb35ec23 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:27.472509 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ccae0-8716-4ac7-bbd0-299f1e8ffb69 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:27.474525 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-9b0ccae0-8716-4ac7-bbd0-299f1e8ffb69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:27.567611 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:56:27.568387 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1329/2689] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:56:27 2022] POST /identity/v3/users => generated 291 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:56:27.573688 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7dcf6b-44cc-432e-a4ad-051fbb35ec23 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:27.581317 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ccae0-8716-4ac7-bbd0-299f1e8ffb69 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:27.589126 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c785d9-346f-4eb3-bcee-15365f636e19 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8fd0f6e5b144d2ca78b8d96722502a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:27.590789 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c785d9-346f-4eb3-bcee-15365f636e19 service placement] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:27.590789 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c785d9-346f-4eb3-bcee-15365f636e19 service placement] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:27.590789 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c785d9-346f-4eb3-bcee-15365f636e19 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:27.591439 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7dcf6b-44cc-432e-a4ad-051fbb35ec23 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:27.593604 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-29c785d9-346f-4eb3-bcee-15365f636e19 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:27.596641 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38416ca9-5bec-4b48-8adb-cb55c7bda04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:27.597271 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38416ca9-5bec-4b48-8adb-cb55c7bda04c admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:27.597548 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38416ca9-5bec-4b48-8adb-cb55c7bda04c admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:27.597819 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38416ca9-5bec-4b48-8adb-cb55c7bda04c admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:27.598534 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38416ca9-5bec-4b48-8adb-cb55c7bda04c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:27.600165 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38416ca9-5bec-4b48-8adb-cb55c7bda04c admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:27.606713 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c785d9-346f-4eb3-bcee-15365f636e19 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:27.608326 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c785d9-346f-4eb3-bcee-15365f636e19 service placement] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:27.612328 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1330/2690] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:56:27.616424 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38416ca9-5bec-4b48-8adb-cb55c7bda04c admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:27.620017 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1361/2691] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:56:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:56:27.629563 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38416ca9-5bec-4b48-8adb-cb55c7bda04c admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:27.634191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6c0606-e207-4da3-b7f0-77b5f4451fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:27.635480 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c0606-e207-4da3-b7f0-77b5f4451fb6 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:27.635779 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c0606-e207-4da3-b7f0-77b5f4451fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:27.635951 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c0606-e207-4da3-b7f0-77b5f4451fb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:27.636594 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6c0606-e207-4da3-b7f0-77b5f4451fb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:27.638299 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6c0606-e207-4da3-b7f0-77b5f4451fb6 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:27.648909 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1331/2692] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:56:27.655019 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c785d9-346f-4eb3-bcee-15365f636e19 service placement] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:27.666291 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c785d9-346f-4eb3-bcee-15365f636e19 service placement] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:27.671246 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c93fc95-941c-405c-b993-df5e88439ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:27.672127 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93fc95-941c-405c-b993-df5e88439ab2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:27.672602 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93fc95-941c-405c-b993-df5e88439ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:27.673131 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93fc95-941c-405c-b993-df5e88439ab2 admin admin] PATH_INFO: `/v3/projects/90836dd8077a4646be6856de7e362fa6/users/4b971e330f56460cb9e7f1bcc34d0115/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:27.699785 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c93fc95-941c-405c-b993-df5e88439ab2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90836dd8077a4646be6856de7e362fa6, user_id=4b971e330f56460cb9e7f1bcc34d0115, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:27.699785 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c93fc95-941c-405c-b993-df5e88439ab2 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:27.860308 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1362/2693] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:56:27 2022] PUT /identity/v3/projects/90836dd8077a4646be6856de7e362fa6/users/4b971e330f56460cb9e7f1bcc34d0115/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:56:27.872573 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4a3e9-4891-4ae1-94dc-8cc36adbb1bd None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:27.872901 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4a3e9-4891-4ae1-94dc-8cc36adbb1bd None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:27.873181 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4a3e9-4891-4ae1-94dc-8cc36adbb1bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:27.902043 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-faa4a3e9-4891-4ae1-94dc-8cc36adbb1bd None None] MFA Rules not processed for user `4b971e330f56460cb9e7f1bcc34d0115`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:56:27.903274 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-faa4a3e9-4891-4ae1-94dc-8cc36adbb1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:27.971713 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1332/2694] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:56:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:56:27.976420 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c93fc95-941c-405c-b993-df5e88439ab2 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:28.043277 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c93fc95-941c-405c-b993-df5e88439ab2 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:28.048108 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28409bcd-c1eb-4d3a-9761-8485bc848756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:28.048707 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28409bcd-c1eb-4d3a-9761-8485bc848756 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:28.049091 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28409bcd-c1eb-4d3a-9761-8485bc848756 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:28.049490 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28409bcd-c1eb-4d3a-9761-8485bc848756 admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:28.050140 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28409bcd-c1eb-4d3a-9761-8485bc848756 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:28.052132 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28409bcd-c1eb-4d3a-9761-8485bc848756 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:28.232868 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1363/2695] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:56:27 2022] POST /identity/v3/projects => generated 383 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:56:28.237006 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa4a3e9-4891-4ae1-94dc-8cc36adbb1bd None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:28.249079 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa4a3e9-4891-4ae1-94dc-8cc36adbb1bd None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:28.304007 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb640bf-14a9-48f7-a519-62d19e7ff05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:28.304617 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb640bf-14a9-48f7-a519-62d19e7ff05e admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:28.304935 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb640bf-14a9-48f7-a519-62d19e7ff05e admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:28.305213 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb640bf-14a9-48f7-a519-62d19e7ff05e admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:28.305856 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb640bf-14a9-48f7-a519-62d19e7ff05e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:28.308409 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb640bf-14a9-48f7-a519-62d19e7ff05e admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:28.340077 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:56:28.423652 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1333/2696] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:56:28 2022] POST /identity/v3/users => generated 298 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:56:28.426182 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28409bcd-c1eb-4d3a-9761-8485bc848756 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:28.439665 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28409bcd-c1eb-4d3a-9761-8485bc848756 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:28.442750 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14c080b-029d-4098-a957-3e8131d21eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:28.443359 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c080b-029d-4098-a957-3e8131d21eb7 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:28.443691 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c080b-029d-4098-a957-3e8131d21eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:28.444098 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c080b-029d-4098-a957-3e8131d21eb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:28.444816 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14c080b-029d-4098-a957-3e8131d21eb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:28.446475 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14c080b-029d-4098-a957-3e8131d21eb7 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:28.458172 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1364/2697] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:56:28.462437 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb640bf-14a9-48f7-a519-62d19e7ff05e admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:28.473176 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb640bf-14a9-48f7-a519-62d19e7ff05e admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:28.478640 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6986def-5511-40b6-bb31-6eba6e3813bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:28.479880 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6986def-5511-40b6-bb31-6eba6e3813bf admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:28.479996 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6986def-5511-40b6-bb31-6eba6e3813bf admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:28.480192 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6986def-5511-40b6-bb31-6eba6e3813bf admin admin] PATH_INFO: `/v3/projects/968f0eb069a9447f841d3a7acc1428ac/users/5f98589c115c47a282edf4997f1be5f2/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:28.497145 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6986def-5511-40b6-bb31-6eba6e3813bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=968f0eb069a9447f841d3a7acc1428ac, user_id=5f98589c115c47a282edf4997f1be5f2, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:28.500054 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6986def-5511-40b6-bb31-6eba6e3813bf admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:28.632066 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1334/2698] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:56:28 2022] PUT /identity/v3/projects/968f0eb069a9447f841d3a7acc1428ac/users/5f98589c115c47a282edf4997f1be5f2/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:56:28.636356 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14c080b-029d-4098-a957-3e8131d21eb7 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:28.646891 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14c080b-029d-4098-a957-3e8131d21eb7 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:28.704127 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47eba3d-a83a-4995-8be4-43b71c538c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:28.704894 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47eba3d-a83a-4995-8be4-43b71c538c84 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:28.705185 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47eba3d-a83a-4995-8be4-43b71c538c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:28.705456 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47eba3d-a83a-4995-8be4-43b71c538c84 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:28.706088 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47eba3d-a83a-4995-8be4-43b71c538c84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:28.707659 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47eba3d-a83a-4995-8be4-43b71c538c84 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:28.717928 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1365/2699] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:56:28.722350 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6986def-5511-40b6-bb31-6eba6e3813bf admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:28.733897 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6986def-5511-40b6-bb31-6eba6e3813bf admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:28.743470 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92424ceb-da59-4211-8c6e-197a8f6f77da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:28.743470 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92424ceb-da59-4211-8c6e-197a8f6f77da admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:28.743470 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92424ceb-da59-4211-8c6e-197a8f6f77da admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:28.743470 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92424ceb-da59-4211-8c6e-197a8f6f77da admin admin] PATH_INFO: `/v3/projects/968f0eb069a9447f841d3a7acc1428ac/users/5f98589c115c47a282edf4997f1be5f2/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:28.744039 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92424ceb-da59-4211-8c6e-197a8f6f77da admin admin] RBAC: Authorizing `identity:create_grant(project_id=968f0eb069a9447f841d3a7acc1428ac, user_id=5f98589c115c47a282edf4997f1be5f2, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:28.745781 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92424ceb-da59-4211-8c6e-197a8f6f77da admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:28.756014 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1335/2700] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:56:28 2022] PUT /identity/v3/projects/968f0eb069a9447f841d3a7acc1428ac/users/5f98589c115c47a282edf4997f1be5f2/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:56:28.766838 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f3821-c539-4233-9a26-e5e70b0ba39b None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:28.767130 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f3821-c539-4233-9a26-e5e70b0ba39b None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:28.767413 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f3821-c539-4233-9a26-e5e70b0ba39b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:28.796647 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-f88f3821-c539-4233-9a26-e5e70b0ba39b None None] MFA Rules not processed for user `5f98589c115c47a282edf4997f1be5f2`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:56:28.797882 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-f88f3821-c539-4233-9a26-e5e70b0ba39b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:28.818198 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:56:28.868016 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1366/2701] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:56:28 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:56:28.878917 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92424ceb-da59-4211-8c6e-197a8f6f77da admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:28.880787 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-92424ceb-da59-4211-8c6e-197a8f6f77da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:28.941229 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92424ceb-da59-4211-8c6e-197a8f6f77da admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:28.944988 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85fced7-446c-4018-a1e2-9508512fb920 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:28.945591 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85fced7-446c-4018-a1e2-9508512fb920 service cinder] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:28.945869 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85fced7-446c-4018-a1e2-9508512fb920 service cinder] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:28.946140 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85fced7-446c-4018-a1e2-9508512fb920 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:28.957821 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85fced7-446c-4018-a1e2-9508512fb920 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:28.959255 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85fced7-446c-4018-a1e2-9508512fb920 service cinder] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:28.969369 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1336/2702] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:56:28 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:56:29.075328 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f3821-c539-4233-9a26-e5e70b0ba39b None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:29.084765 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f3821-c539-4233-9a26-e5e70b0ba39b None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:29.089345 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4f7025-5481-4bfc-8688-b543fce88d82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:29.089948 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f7025-5481-4bfc-8688-b543fce88d82 service cinder] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:29.090232 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f7025-5481-4bfc-8688-b543fce88d82 service cinder] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:29.090506 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f7025-5481-4bfc-8688-b543fce88d82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:29.147929 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4f7025-5481-4bfc-8688-b543fce88d82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:29.149383 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4f7025-5481-4bfc-8688-b543fce88d82 service cinder] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:29.158477 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1367/2703] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:56:29 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:56:32.592073 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85fced7-446c-4018-a1e2-9508512fb920 service cinder] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:32.658736 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85fced7-446c-4018-a1e2-9508512fb920 service cinder] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:32.663454 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0448d725-ae32-4ec7-9117-09ba9729438a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:32.664081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0448d725-ae32-4ec7-9117-09ba9729438a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:32.664365 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0448d725-ae32-4ec7-9117-09ba9729438a admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:32.664641 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0448d725-ae32-4ec7-9117-09ba9729438a admin admin] PATH_INFO: `/v3/users/4b971e330f56460cb9e7f1bcc34d0115` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:32.665968 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0448d725-ae32-4ec7-9117-09ba9729438a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b971e330f56460cb9e7f1bcc34d0115)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:32.667887 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0448d725-ae32-4ec7-9117-09ba9729438a admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:32.680606 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:56:32.901794 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-0448d725-ae32-4ec7-9117-09ba9729438a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b971e330f56460cb9e7f1bcc34d0115'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:32.913906 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-0448d725-ae32-4ec7-9117-09ba9729438a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b971e330f56460cb9e7f1bcc34d0115'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:32.922036 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-0448d725-ae32-4ec7-9117-09ba9729438a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b971e330f56460cb9e7f1bcc34d0115'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:32.996410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-0448d725-ae32-4ec7-9117-09ba9729438a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b971e330f56460cb9e7f1bcc34d0115'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:32.997076 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-0448d725-ae32-4ec7-9117-09ba9729438a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b971e330f56460cb9e7f1bcc34d0115'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:33.003575 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1337/2704] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:56:32 2022] DELETE /identity/v3/users/4b971e330f56460cb9e7f1bcc34d0115 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:56:33.008151 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a75c8-950a-438a-921b-9b9f37af24a5 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:33.008612 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a75c8-950a-438a-921b-9b9f37af24a5 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:33.008712 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a75c8-950a-438a-921b-9b9f37af24a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:33.029540 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-967a75c8-950a-438a-921b-9b9f37af24a5 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:56:33.030792 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-967a75c8-950a-438a-921b-9b9f37af24a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:33.090161 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1368/2705] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:56:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:56:33.100211 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0448d725-ae32-4ec7-9117-09ba9729438a admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:33.101693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-0448d725-ae32-4ec7-9117-09ba9729438a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:33.183555 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0448d725-ae32-4ec7-9117-09ba9729438a admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:33.187183 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4480a3d-50af-4d7f-80ca-709c7890c93f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:33.187791 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4480a3d-50af-4d7f-80ca-709c7890c93f service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:33.188073 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4480a3d-50af-4d7f-80ca-709c7890c93f service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:33.188350 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4480a3d-50af-4d7f-80ca-709c7890c93f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:33.198503 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4480a3d-50af-4d7f-80ca-709c7890c93f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:33.199962 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4480a3d-50af-4d7f-80ca-709c7890c93f service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:33.209600 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1338/2706] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:56:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:56:33.668762 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a75c8-950a-438a-921b-9b9f37af24a5 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:33.670894 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-967a75c8-950a-438a-921b-9b9f37af24a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:33.683925 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a75c8-950a-438a-921b-9b9f37af24a5 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:33.689876 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae18316-3849-4a81-9ec1-ec6483da58b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:33.690513 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae18316-3849-4a81-9ec1-ec6483da58b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:33.690799 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae18316-3849-4a81-9ec1-ec6483da58b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:33.691075 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae18316-3849-4a81-9ec1-ec6483da58b0 admin admin] PATH_INFO: `/v3/projects/90836dd8077a4646be6856de7e362fa6` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:33.692218 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae18316-3849-4a81-9ec1-ec6483da58b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90836dd8077a4646be6856de7e362fa6)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:33.694184 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae18316-3849-4a81-9ec1-ec6483da58b0 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:33.808913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-eae18316-3849-4a81-9ec1-ec6483da58b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90836dd8077a4646be6856de7e362fa6', 'request_id': 'req-eae18316-3849-4a81-9ec1-ec6483da58b0'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:33.810581 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:56:33.815888 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-eae18316-3849-4a81-9ec1-ec6483da58b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90836dd8077a4646be6856de7e362fa6', 'request_id': 'req-eae18316-3849-4a81-9ec1-ec6483da58b0'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:33.824000 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-eae18316-3849-4a81-9ec1-ec6483da58b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90836dd8077a4646be6856de7e362fa6', 'request_id': 'req-eae18316-3849-4a81-9ec1-ec6483da58b0'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:33.909888 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-eae18316-3849-4a81-9ec1-ec6483da58b0 admin admin] The token cache is being invalidate because project 90836dd8077a4646be6856de7e362fa6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:56:33.910170 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-eae18316-3849-4a81-9ec1-ec6483da58b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:33.910800 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-eae18316-3849-4a81-9ec1-ec6483da58b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:33.912269 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1369/2707] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:56:33 2022] DELETE /identity/v3/projects/90836dd8077a4646be6856de7e362fa6 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:56:33.916531 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4480a3d-50af-4d7f-80ca-709c7890c93f service neutron] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:33.926686 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4480a3d-50af-4d7f-80ca-709c7890c93f service neutron] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:33.989208 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42631a9-8ece-4569-8eed-10a54c2f88dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:33.989850 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42631a9-8ece-4569-8eed-10a54c2f88dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:33.990136 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42631a9-8ece-4569-8eed-10a54c2f88dc admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:33.990424 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42631a9-8ece-4569-8eed-10a54c2f88dc admin admin] PATH_INFO: `/v3/users/5f98589c115c47a282edf4997f1be5f2` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:33.991699 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42631a9-8ece-4569-8eed-10a54c2f88dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f98589c115c47a282edf4997f1be5f2)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:33.994286 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42631a9-8ece-4569-8eed-10a54c2f88dc admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:34.010127 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:56:34.165206 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-f42631a9-8ece-4569-8eed-10a54c2f88dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f98589c115c47a282edf4997f1be5f2'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:34.182197 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-f42631a9-8ece-4569-8eed-10a54c2f88dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f98589c115c47a282edf4997f1be5f2'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:34.190296 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-f42631a9-8ece-4569-8eed-10a54c2f88dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f98589c115c47a282edf4997f1be5f2'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:34.297763 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-f42631a9-8ece-4569-8eed-10a54c2f88dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f98589c115c47a282edf4997f1be5f2'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:34.298404 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-f42631a9-8ece-4569-8eed-10a54c2f88dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f98589c115c47a282edf4997f1be5f2'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:34.304997 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1339/2708] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:56:33 2022] DELETE /identity/v3/users/5f98589c115c47a282edf4997f1be5f2 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:56:34.758795 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae18316-3849-4a81-9ec1-ec6483da58b0 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:34.760952 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-eae18316-3849-4a81-9ec1-ec6483da58b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:34.776614 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae18316-3849-4a81-9ec1-ec6483da58b0 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:34.837894 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bc56e7-a9ec-4286-9d37-bd40b236b167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:34.838533 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc56e7-a9ec-4286-9d37-bd40b236b167 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:34.841505 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc56e7-a9ec-4286-9d37-bd40b236b167 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:34.841505 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc56e7-a9ec-4286-9d37-bd40b236b167 admin admin] PATH_INFO: `/v3/projects/968f0eb069a9447f841d3a7acc1428ac` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:34.841505 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bc56e7-a9ec-4286-9d37-bd40b236b167 admin admin] RBAC: Authorizing `identity:delete_project(project_id=968f0eb069a9447f841d3a7acc1428ac)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:34.842102 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bc56e7-a9ec-4286-9d37-bd40b236b167 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:34.941131 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-77bc56e7-a9ec-4286-9d37-bd40b236b167 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '968f0eb069a9447f841d3a7acc1428ac', 'request_id': 'req-77bc56e7-a9ec-4286-9d37-bd40b236b167'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:34.947204 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-77bc56e7-a9ec-4286-9d37-bd40b236b167 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '968f0eb069a9447f841d3a7acc1428ac', 'request_id': 'req-77bc56e7-a9ec-4286-9d37-bd40b236b167'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:34.953812 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-77bc56e7-a9ec-4286-9d37-bd40b236b167 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '968f0eb069a9447f841d3a7acc1428ac', 'request_id': 'req-77bc56e7-a9ec-4286-9d37-bd40b236b167'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:35.023236 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-77bc56e7-a9ec-4286-9d37-bd40b236b167 admin admin] The token cache is being invalidate because project 968f0eb069a9447f841d3a7acc1428ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:56:35.023463 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-77bc56e7-a9ec-4286-9d37-bd40b236b167 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:35.024046 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-77bc56e7-a9ec-4286-9d37-bd40b236b167 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:35.025591 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1370/2709] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:56:34 2022] DELETE /identity/v3/projects/968f0eb069a9447f841d3a7acc1428ac => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:56:35.031013 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0734aa1-d319-46c6-98a2-67cbe05bf700 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:35.032302 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0734aa1-d319-46c6-98a2-67cbe05bf700 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:35.032302 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0734aa1-d319-46c6-98a2-67cbe05bf700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:35.055478 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-d0734aa1-d319-46c6-98a2-67cbe05bf700 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:56:35.057831 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-d0734aa1-d319-46c6-98a2-67cbe05bf700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:35.139283 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1340/2710] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:56:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:56:35.147921 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf45b4f8-67b9-4af4-b09c-8a2ce5528e8e None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:35.148203 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf45b4f8-67b9-4af4-b09c-8a2ce5528e8e None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:35.148476 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf45b4f8-67b9-4af4-b09c-8a2ce5528e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:35.169022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-cf45b4f8-67b9-4af4-b09c-8a2ce5528e8e None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:56:35.170227 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-cf45b4f8-67b9-4af4-b09c-8a2ce5528e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:35.178205 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1371/2711] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:56:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:56:35.185352 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0734aa1-d319-46c6-98a2-67cbe05bf700 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:35.199549 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0734aa1-d319-46c6-98a2-67cbe05bf700 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:35.204228 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9472fe9-383b-45b4-9c4c-3a7a239b3cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:35.204854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9472fe9-383b-45b4-9c4c-3a7a239b3cf8 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:35.205136 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9472fe9-383b-45b4-9c4c-3a7a239b3cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:35.205406 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9472fe9-383b-45b4-9c4c-3a7a239b3cf8 admin admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:35.206042 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9472fe9-383b-45b4-9c4c-3a7a239b3cf8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:56:35.206344 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9472fe9-383b-45b4-9c4c-3a7a239b3cf8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:35.207930 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9472fe9-383b-45b4-9c4c-3a7a239b3cf8 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:35.220374 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1341/2712] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:56:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:56:35.224758 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf45b4f8-67b9-4af4-b09c-8a2ce5528e8e None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:35.234029 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf45b4f8-67b9-4af4-b09c-8a2ce5528e8e None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:35.238672 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabf84ea-9765-4bd8-b6d1-13c93cb75306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:35.239270 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf84ea-9765-4bd8-b6d1-13c93cb75306 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:35.239548 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf84ea-9765-4bd8-b6d1-13c93cb75306 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:35.239820 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf84ea-9765-4bd8-b6d1-13c93cb75306 admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:35.240451 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabf84ea-9765-4bd8-b6d1-13c93cb75306 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:35.242378 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabf84ea-9765-4bd8-b6d1-13c93cb75306 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:35.334485 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1372/2713] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:56:35 2022] POST /identity/v3/projects => generated 385 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:56:35.339363 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9472fe9-383b-45b4-9c4c-3a7a239b3cf8 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:35.349686 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9472fe9-383b-45b4-9c4c-3a7a239b3cf8 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:35.409163 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea728f0-a9bb-4c7e-bddd-3204cfa503d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:35.409924 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea728f0-a9bb-4c7e-bddd-3204cfa503d4 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:35.410416 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea728f0-a9bb-4c7e-bddd-3204cfa503d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:35.410997 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea728f0-a9bb-4c7e-bddd-3204cfa503d4 admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:35.413152 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea728f0-a9bb-4c7e-bddd-3204cfa503d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:35.415407 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea728f0-a9bb-4c7e-bddd-3204cfa503d4 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:35.504531 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1342/2714] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:56:35 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:56:35.508765 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabf84ea-9765-4bd8-b6d1-13c93cb75306 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:35.520936 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabf84ea-9765-4bd8-b6d1-13c93cb75306 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:35.526030 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2734dd9-b8fb-47d9-b859-72ce9274ba76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:35.526658 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2734dd9-b8fb-47d9-b859-72ce9274ba76 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:35.526938 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2734dd9-b8fb-47d9-b859-72ce9274ba76 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:35.527214 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2734dd9-b8fb-47d9-b859-72ce9274ba76 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:35.527865 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2734dd9-b8fb-47d9-b859-72ce9274ba76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:35.529672 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2734dd9-b8fb-47d9-b859-72ce9274ba76 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:35.540677 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1373/2715] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:56:35.546861 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea728f0-a9bb-4c7e-bddd-3204cfa503d4 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:35.555276 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea728f0-a9bb-4c7e-bddd-3204cfa503d4 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:35.560807 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff58164-afba-4593-930f-c1151e555f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:35.561404 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff58164-afba-4593-930f-c1151e555f8a admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:35.561688 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff58164-afba-4593-930f-c1151e555f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:35.561966 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff58164-afba-4593-930f-c1151e555f8a admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:35.562610 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff58164-afba-4593-930f-c1151e555f8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:35.564597 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff58164-afba-4593-930f-c1151e555f8a admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:35.566955 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:56:35.576972 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1343/2716] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:56:35.581122 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2734dd9-b8fb-47d9-b859-72ce9274ba76 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:35.591541 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2734dd9-b8fb-47d9-b859-72ce9274ba76 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:35.596251 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a5157b-306c-4eeb-aebd-a9416cd2b22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:35.596939 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a5157b-306c-4eeb-aebd-a9416cd2b22c admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:35.597225 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a5157b-306c-4eeb-aebd-a9416cd2b22c admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:35.597499 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a5157b-306c-4eeb-aebd-a9416cd2b22c admin admin] PATH_INFO: `/v3/projects/244f87ad3c9842a3a9723603f9da63d5/users/8a02520834d948feaee61654945f2fc8/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:35.612964 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a5157b-306c-4eeb-aebd-a9416cd2b22c admin admin] RBAC: Authorizing `identity:create_grant(project_id=244f87ad3c9842a3a9723603f9da63d5, user_id=8a02520834d948feaee61654945f2fc8, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:35.615643 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a5157b-306c-4eeb-aebd-a9416cd2b22c admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:35.699858 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1374/2717] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:56:35 2022] PUT /identity/v3/projects/244f87ad3c9842a3a9723603f9da63d5/users/8a02520834d948feaee61654945f2fc8/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:56:35.710504 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72845a8-8fb1-435e-becc-2ab285c1edd1 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:35.711060 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72845a8-8fb1-435e-becc-2ab285c1edd1 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:35.711342 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72845a8-8fb1-435e-becc-2ab285c1edd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:35.740367 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-b72845a8-8fb1-435e-becc-2ab285c1edd1 None None] MFA Rules not processed for user `8a02520834d948feaee61654945f2fc8`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:56:35.741595 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-b72845a8-8fb1-435e-becc-2ab285c1edd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:35.808300 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1344/2718] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:56:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:56:35.813003 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a5157b-306c-4eeb-aebd-a9416cd2b22c admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:35.818602 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:56:35.880245 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a5157b-306c-4eeb-aebd-a9416cd2b22c admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:35.884830 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74060aae-78a3-493c-a0cc-58ef4df75d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:35.885442 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74060aae-78a3-493c-a0cc-58ef4df75d38 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:35.885714 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74060aae-78a3-493c-a0cc-58ef4df75d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:35.885998 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74060aae-78a3-493c-a0cc-58ef4df75d38 admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:35.886625 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74060aae-78a3-493c-a0cc-58ef4df75d38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:35.891253 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74060aae-78a3-493c-a0cc-58ef4df75d38 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:36.009762 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1375/2719] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:56:35 2022] POST /identity/v3/projects => generated 389 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:56:36.013709 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72845a8-8fb1-435e-becc-2ab285c1edd1 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:36.029209 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72845a8-8fb1-435e-becc-2ab285c1edd1 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:36.086012 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e25a1dd-8136-483b-9657-76c6282006e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:36.086664 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e25a1dd-8136-483b-9657-76c6282006e7 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:36.086873 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e25a1dd-8136-483b-9657-76c6282006e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:36.087145 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e25a1dd-8136-483b-9657-76c6282006e7 admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:36.088039 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e25a1dd-8136-483b-9657-76c6282006e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:36.090004 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e25a1dd-8136-483b-9657-76c6282006e7 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:36.225254 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1345/2720] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:56:36 2022] POST /identity/v3/users => generated 301 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:56:36.230477 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74060aae-78a3-493c-a0cc-58ef4df75d38 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:36.241174 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74060aae-78a3-493c-a0cc-58ef4df75d38 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:36.246297 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffabbc1b-d4e4-49cf-8798-09fe128961fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:36.246888 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffabbc1b-d4e4-49cf-8798-09fe128961fb admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:36.247164 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffabbc1b-d4e4-49cf-8798-09fe128961fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:36.247433 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffabbc1b-d4e4-49cf-8798-09fe128961fb admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:36.248069 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffabbc1b-d4e4-49cf-8798-09fe128961fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:36.249680 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffabbc1b-d4e4-49cf-8798-09fe128961fb admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:36.260868 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1376/2721] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:56:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:56:36.268515 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e25a1dd-8136-483b-9657-76c6282006e7 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:36.277093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e25a1dd-8136-483b-9657-76c6282006e7 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:36.281994 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42081e5-0e6d-424c-ab97-6a5623af7714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:36.282652 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42081e5-0e6d-424c-ab97-6a5623af7714 admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:36.283029 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42081e5-0e6d-424c-ab97-6a5623af7714 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:36.283306 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42081e5-0e6d-424c-ab97-6a5623af7714 admin admin] PATH_INFO: `/v3/projects/b3b1087355e04d5095884d37bf831eb3/users/f49d1a7292314bbbbc3872e6592f8b1e/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:36.301274 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42081e5-0e6d-424c-ab97-6a5623af7714 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3b1087355e04d5095884d37bf831eb3, user_id=f49d1a7292314bbbbc3872e6592f8b1e, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:36.303927 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42081e5-0e6d-424c-ab97-6a5623af7714 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:36.413956 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1346/2722] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:56:36 2022] PUT /identity/v3/projects/b3b1087355e04d5095884d37bf831eb3/users/f49d1a7292314bbbbc3872e6592f8b1e/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:56:36.420564 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffabbc1b-d4e4-49cf-8798-09fe128961fb admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:36.431071 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffabbc1b-d4e4-49cf-8798-09fe128961fb admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:36.491042 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe3d89-fc6e-44c9-bc2b-779cb177378f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:36.491566 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe3d89-fc6e-44c9-bc2b-779cb177378f admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:36.491843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe3d89-fc6e-44c9-bc2b-779cb177378f admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:36.492112 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe3d89-fc6e-44c9-bc2b-779cb177378f admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:36.492803 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fe3d89-fc6e-44c9-bc2b-779cb177378f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:36.494475 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fe3d89-fc6e-44c9-bc2b-779cb177378f admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:36.504434 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1377/2723] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:56:36 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:56:36.508304 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42081e5-0e6d-424c-ab97-6a5623af7714 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:36.519765 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42081e5-0e6d-424c-ab97-6a5623af7714 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:36.525187 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694923ae-86d7-4248-9aef-a6491777f7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:36.526395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694923ae-86d7-4248-9aef-a6491777f7bf admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:36.527173 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694923ae-86d7-4248-9aef-a6491777f7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:36.527173 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694923ae-86d7-4248-9aef-a6491777f7bf admin admin] PATH_INFO: `/v3/projects/b3b1087355e04d5095884d37bf831eb3/users/f49d1a7292314bbbbc3872e6592f8b1e/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:36.529059 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694923ae-86d7-4248-9aef-a6491777f7bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3b1087355e04d5095884d37bf831eb3, user_id=f49d1a7292314bbbbc3872e6592f8b1e, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:36.531747 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694923ae-86d7-4248-9aef-a6491777f7bf admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:36.549710 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1347/2724] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:56:36 2022] PUT /identity/v3/projects/b3b1087355e04d5095884d37bf831eb3/users/f49d1a7292314bbbbc3872e6592f8b1e/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:56:36.561459 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d6db9-2848-4af6-81a5-c5f7d15c76de None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:36.566378 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d6db9-2848-4af6-81a5-c5f7d15c76de None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:36.566378 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d6db9-2848-4af6-81a5-c5f7d15c76de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:36.596897 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-ea1d6db9-2848-4af6-81a5-c5f7d15c76de None None] MFA Rules not processed for user `f49d1a7292314bbbbc3872e6592f8b1e`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:56:36.598244 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-ea1d6db9-2848-4af6-81a5-c5f7d15c76de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:36.670902 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:56:36.674436 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1378/2725] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:56:36 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:56:36.685229 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694923ae-86d7-4248-9aef-a6491777f7bf admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:36.686967 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-694923ae-86d7-4248-9aef-a6491777f7bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:36.740562 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:56:36.755640 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694923ae-86d7-4248-9aef-a6491777f7bf admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:36.760399 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53cb7d3-08c4-4d61-9eca-3523bb7caaff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:36.761079 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53cb7d3-08c4-4d61-9eca-3523bb7caaff service cinder] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:36.761356 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53cb7d3-08c4-4d61-9eca-3523bb7caaff service cinder] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:36.761655 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53cb7d3-08c4-4d61-9eca-3523bb7caaff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:36.770959 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53cb7d3-08c4-4d61-9eca-3523bb7caaff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:36.772389 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53cb7d3-08c4-4d61-9eca-3523bb7caaff service cinder] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:36.782543 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1348/2726] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:56:36 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:56:36.929050 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1d6db9-2848-4af6-81a5-c5f7d15c76de None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:36.996054 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1d6db9-2848-4af6-81a5-c5f7d15c76de None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:37.000629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f2075-4884-49cb-a682-7e1b9bcafd1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:37.001295 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f2075-4884-49cb-a682-7e1b9bcafd1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:37.001581 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f2075-4884-49cb-a682-7e1b9bcafd1c admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:37.001856 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f2075-4884-49cb-a682-7e1b9bcafd1c admin admin] PATH_INFO: `/v3/users/8a02520834d948feaee61654945f2fc8` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:37.003022 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539f2075-4884-49cb-a682-7e1b9bcafd1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a02520834d948feaee61654945f2fc8)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:37.005123 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539f2075-4884-49cb-a682-7e1b9bcafd1c admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:37.296441 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-539f2075-4884-49cb-a682-7e1b9bcafd1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a02520834d948feaee61654945f2fc8'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:37.369057 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-539f2075-4884-49cb-a682-7e1b9bcafd1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a02520834d948feaee61654945f2fc8'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:37.383667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-539f2075-4884-49cb-a682-7e1b9bcafd1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a02520834d948feaee61654945f2fc8'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:37.390658 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-539f2075-4884-49cb-a682-7e1b9bcafd1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a02520834d948feaee61654945f2fc8'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:37.391379 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-539f2075-4884-49cb-a682-7e1b9bcafd1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a02520834d948feaee61654945f2fc8'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:37.397440 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1379/2727] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:56:36 2022] DELETE /identity/v3/users/8a02520834d948feaee61654945f2fc8 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:56:37.402132 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58c506-58df-4297-95ab-536b9fd60270 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:37.402765 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58c506-58df-4297-95ab-536b9fd60270 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:37.402765 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58c506-58df-4297-95ab-536b9fd60270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:37.422856 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-ae58c506-58df-4297-95ab-536b9fd60270 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:56:37.424071 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-ae58c506-58df-4297-95ab-536b9fd60270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:37.484858 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1349/2728] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:56:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:56:37.495454 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f2075-4884-49cb-a682-7e1b9bcafd1c admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:37.496936 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-539f2075-4884-49cb-a682-7e1b9bcafd1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:37.570557 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f2075-4884-49cb-a682-7e1b9bcafd1c admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:37.574311 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264134e5-e5e5-4f57-89b9-1e11bc2f7706 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:37.574938 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264134e5-e5e5-4f57-89b9-1e11bc2f7706 service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:37.575398 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264134e5-e5e5-4f57-89b9-1e11bc2f7706 service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:37.575678 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264134e5-e5e5-4f57-89b9-1e11bc2f7706 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:37.585213 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264134e5-e5e5-4f57-89b9-1e11bc2f7706 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:37.586685 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264134e5-e5e5-4f57-89b9-1e11bc2f7706 service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:37.596526 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1380/2729] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:56:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:56:38.448892 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae58c506-58df-4297-95ab-536b9fd60270 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:38.451293 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-ae58c506-58df-4297-95ab-536b9fd60270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:38.465298 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae58c506-58df-4297-95ab-536b9fd60270 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:38.470882 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5d8ba-0117-4f4a-9295-893ab0393fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:38.471732 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5d8ba-0117-4f4a-9295-893ab0393fa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:38.471957 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5d8ba-0117-4f4a-9295-893ab0393fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:38.472299 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5d8ba-0117-4f4a-9295-893ab0393fa9 admin admin] PATH_INFO: `/v3/projects/244f87ad3c9842a3a9723603f9da63d5` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:38.474100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a5d8ba-0117-4f4a-9295-893ab0393fa9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=244f87ad3c9842a3a9723603f9da63d5)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:38.476464 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a5d8ba-0117-4f4a-9295-893ab0393fa9 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:38.678912 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-28a5d8ba-0117-4f4a-9295-893ab0393fa9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '244f87ad3c9842a3a9723603f9da63d5', 'request_id': 'req-28a5d8ba-0117-4f4a-9295-893ab0393fa9'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:38.807583 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-28a5d8ba-0117-4f4a-9295-893ab0393fa9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '244f87ad3c9842a3a9723603f9da63d5', 'request_id': 'req-28a5d8ba-0117-4f4a-9295-893ab0393fa9'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:38.815954 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-28a5d8ba-0117-4f4a-9295-893ab0393fa9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '244f87ad3c9842a3a9723603f9da63d5', 'request_id': 'req-28a5d8ba-0117-4f4a-9295-893ab0393fa9'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:38.838865 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-28a5d8ba-0117-4f4a-9295-893ab0393fa9 admin admin] The token cache is being invalidate because project 244f87ad3c9842a3a9723603f9da63d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:56:38.839302 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-28a5d8ba-0117-4f4a-9295-893ab0393fa9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:38.840389 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-28a5d8ba-0117-4f4a-9295-893ab0393fa9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:38.842958 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1350/2730] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:56:38 2022] DELETE /identity/v3/projects/244f87ad3c9842a3a9723603f9da63d5 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:56:38.847451 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264134e5-e5e5-4f57-89b9-1e11bc2f7706 service neutron] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:38.858624 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264134e5-e5e5-4f57-89b9-1e11bc2f7706 service neutron] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:38.881128 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:56:38.924604 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd6bc46-7bf5-432a-8bc8-1943aaaed4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:38.925291 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd6bc46-7bf5-432a-8bc8-1943aaaed4ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:38.925573 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd6bc46-7bf5-432a-8bc8-1943aaaed4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:38.925851 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd6bc46-7bf5-432a-8bc8-1943aaaed4ce admin admin] PATH_INFO: `/v3/users/f49d1a7292314bbbbc3872e6592f8b1e` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:38.927131 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd6bc46-7bf5-432a-8bc8-1943aaaed4ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=f49d1a7292314bbbbc3872e6592f8b1e)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:38.929222 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd6bc46-7bf5-432a-8bc8-1943aaaed4ce admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:39.158894 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-cfd6bc46-7bf5-432a-8bc8-1943aaaed4ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f49d1a7292314bbbbc3872e6592f8b1e'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:39.253270 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-cfd6bc46-7bf5-432a-8bc8-1943aaaed4ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f49d1a7292314bbbbc3872e6592f8b1e'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:39.267563 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-cfd6bc46-7bf5-432a-8bc8-1943aaaed4ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f49d1a7292314bbbbc3872e6592f8b1e'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:39.274903 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-cfd6bc46-7bf5-432a-8bc8-1943aaaed4ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f49d1a7292314bbbbc3872e6592f8b1e'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:39.276290 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-cfd6bc46-7bf5-432a-8bc8-1943aaaed4ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f49d1a7292314bbbbc3872e6592f8b1e'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:39.284920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1381/2731] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:56:38 2022] DELETE /identity/v3/users/f49d1a7292314bbbbc3872e6592f8b1e => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:56:39.776486 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5d8ba-0117-4f4a-9295-893ab0393fa9 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:39.778285 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-28a5d8ba-0117-4f4a-9295-893ab0393fa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:39.792544 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5d8ba-0117-4f4a-9295-893ab0393fa9 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:39.857193 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c1bda-c5f7-43bb-bc02-af188869ef09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:39.857831 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c1bda-c5f7-43bb-bc02-af188869ef09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:39.858115 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c1bda-c5f7-43bb-bc02-af188869ef09 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:39.858390 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c1bda-c5f7-43bb-bc02-af188869ef09 admin admin] PATH_INFO: `/v3/projects/b3b1087355e04d5095884d37bf831eb3` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:39.859520 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85c1bda-c5f7-43bb-bc02-af188869ef09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3b1087355e04d5095884d37bf831eb3)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:39.861478 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85c1bda-c5f7-43bb-bc02-af188869ef09 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:39.870441 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:56:39.974746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-c85c1bda-c5f7-43bb-bc02-af188869ef09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3b1087355e04d5095884d37bf831eb3', 'request_id': 'req-c85c1bda-c5f7-43bb-bc02-af188869ef09'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:40.047540 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-c85c1bda-c5f7-43bb-bc02-af188869ef09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3b1087355e04d5095884d37bf831eb3', 'request_id': 'req-c85c1bda-c5f7-43bb-bc02-af188869ef09'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:40.053753 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-c85c1bda-c5f7-43bb-bc02-af188869ef09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3b1087355e04d5095884d37bf831eb3', 'request_id': 'req-c85c1bda-c5f7-43bb-bc02-af188869ef09'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:40.066864 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-c85c1bda-c5f7-43bb-bc02-af188869ef09 admin admin] The token cache is being invalidate because project b3b1087355e04d5095884d37bf831eb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:56:40.067151 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-c85c1bda-c5f7-43bb-bc02-af188869ef09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:40.067828 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-c85c1bda-c5f7-43bb-bc02-af188869ef09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:40.069544 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1351/2732] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:56:39 2022] DELETE /identity/v3/projects/b3b1087355e04d5095884d37bf831eb3 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:56:40.075231 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079df60-c444-40fe-b688-d68279459b3f None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:40.075783 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079df60-c444-40fe-b688-d68279459b3f None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:40.075783 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079df60-c444-40fe-b688-d68279459b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:40.095582 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-7079df60-c444-40fe-b688-d68279459b3f None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:56:40.096798 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-7079df60-c444-40fe-b688-d68279459b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:40.156752 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1382/2733] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:56:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:56:40.224781 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf0867-f8cf-4caf-9769-3353baf8c1f3 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:40.225285 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf0867-f8cf-4caf-9769-3353baf8c1f3 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:40.225630 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf0867-f8cf-4caf-9769-3353baf8c1f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:40.250669 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-8bbf0867-f8cf-4caf-9769-3353baf8c1f3 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:56:40.253436 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-8bbf0867-f8cf-4caf-9769-3353baf8c1f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:40.261013 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1352/2734] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:56:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:56:40.265778 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7079df60-c444-40fe-b688-d68279459b3f None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:40.277822 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7079df60-c444-40fe-b688-d68279459b3f None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:40.284275 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4b03d7-b6ff-4b0f-a67a-352b8fa72eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:40.285110 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b03d7-b6ff-4b0f-a67a-352b8fa72eb2 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:40.285548 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b03d7-b6ff-4b0f-a67a-352b8fa72eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:40.285938 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b03d7-b6ff-4b0f-a67a-352b8fa72eb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:40.286840 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4b03d7-b6ff-4b0f-a67a-352b8fa72eb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:56:40.287215 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4b03d7-b6ff-4b0f-a67a-352b8fa72eb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:40.290135 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4b03d7-b6ff-4b0f-a67a-352b8fa72eb2 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:40.304808 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1383/2735] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:56:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:56:40.309703 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf0867-f8cf-4caf-9769-3353baf8c1f3 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:40.319321 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf0867-f8cf-4caf-9769-3353baf8c1f3 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:40.325333 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40431bc-bade-48c5-8112-f72ee3e421e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:40.325987 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40431bc-bade-48c5-8112-f72ee3e421e1 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:40.326342 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40431bc-bade-48c5-8112-f72ee3e421e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:40.326621 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40431bc-bade-48c5-8112-f72ee3e421e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:40.327271 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40431bc-bade-48c5-8112-f72ee3e421e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:40.329207 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40431bc-bade-48c5-8112-f72ee3e421e1 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:40.381861 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f35a4e2-fcd3-4a4e-bef2-63e5716124b1 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:40.382225 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f35a4e2-fcd3-4a4e-bef2-63e5716124b1 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:40.382517 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f35a4e2-fcd3-4a4e-bef2-63e5716124b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:40.403679 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-0f35a4e2-fcd3-4a4e-bef2-63e5716124b1 None None] MFA Rules not processed for user `f3121e61f02e4869b8214a5caf1b580a`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:56:40.404932 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-0f35a4e2-fcd3-4a4e-bef2-63e5716124b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:40.429652 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:56:40.461244 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1353/2736] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:56:40 2022] POST /identity/v3/projects => generated 385 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:56:40.466358 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40431bc-bade-48c5-8112-f72ee3e421e1 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:40.472024 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1384/2737] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:56:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:56:40.545170 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40431bc-bade-48c5-8112-f72ee3e421e1 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:40.551484 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302b9618-017c-4bbb-9746-a1b545ac5cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:40.552166 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b9618-017c-4bbb-9746-a1b545ac5cd3 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:40.552449 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b9618-017c-4bbb-9746-a1b545ac5cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:40.552744 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b9618-017c-4bbb-9746-a1b545ac5cd3 admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:40.553374 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302b9618-017c-4bbb-9746-a1b545ac5cd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:40.556158 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302b9618-017c-4bbb-9746-a1b545ac5cd3 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:40.665773 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1354/2738] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:56:40 2022] POST /identity/v3/users => generated 293 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:56:40.669982 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f35a4e2-fcd3-4a4e-bef2-63e5716124b1 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:40.682662 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f35a4e2-fcd3-4a4e-bef2-63e5716124b1 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:40.687824 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7b0b7c-79d4-437b-86d9-9af844e09d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:40.688498 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b0b7c-79d4-437b-86d9-9af844e09d87 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:40.688859 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b0b7c-79d4-437b-86d9-9af844e09d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:40.689179 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b0b7c-79d4-437b-86d9-9af844e09d87 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:40.689868 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7b0b7c-79d4-437b-86d9-9af844e09d87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:40.691604 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7b0b7c-79d4-437b-86d9-9af844e09d87 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:40.709199 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1385/2739] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:56:40.714552 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302b9618-017c-4bbb-9746-a1b545ac5cd3 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:40.724949 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302b9618-017c-4bbb-9746-a1b545ac5cd3 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:40.731446 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e100d2-b1f7-4dc3-a3eb-a9baffd357ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:40.732232 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e100d2-b1f7-4dc3-a3eb-a9baffd357ea admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:40.732512 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e100d2-b1f7-4dc3-a3eb-a9baffd357ea admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:40.733202 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e100d2-b1f7-4dc3-a3eb-a9baffd357ea admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:40.734164 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e100d2-b1f7-4dc3-a3eb-a9baffd357ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:40.736201 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e100d2-b1f7-4dc3-a3eb-a9baffd357ea admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:40.747519 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1355/2740] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:56:40.751754 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7b0b7c-79d4-437b-86d9-9af844e09d87 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:40.763406 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7b0b7c-79d4-437b-86d9-9af844e09d87 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:40.769245 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3930cf-9e7c-496e-a46e-54bc8d857101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:40.770085 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3930cf-9e7c-496e-a46e-54bc8d857101 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:40.770470 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3930cf-9e7c-496e-a46e-54bc8d857101 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:40.770756 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3930cf-9e7c-496e-a46e-54bc8d857101 admin admin] PATH_INFO: `/v3/projects/e1accf0779de46c78b1e93a180b479e9/users/5a5e267509c948fd9fadb3845e1b984a/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:40.790376 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3930cf-9e7c-496e-a46e-54bc8d857101 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1accf0779de46c78b1e93a180b479e9, user_id=5a5e267509c948fd9fadb3845e1b984a, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:40.793925 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3930cf-9e7c-496e-a46e-54bc8d857101 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:40.862068 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1386/2741] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:56:40 2022] PUT /identity/v3/projects/e1accf0779de46c78b1e93a180b479e9/users/5a5e267509c948fd9fadb3845e1b984a/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:56:40.872353 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc7c76-6e46-49e2-90a2-75ed7200108d None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:40.873132 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc7c76-6e46-49e2-90a2-75ed7200108d None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:40.873132 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc7c76-6e46-49e2-90a2-75ed7200108d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:40.906697 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-50cc7c76-6e46-49e2-90a2-75ed7200108d None None] MFA Rules not processed for user `5a5e267509c948fd9fadb3845e1b984a`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:56:40.908032 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-50cc7c76-6e46-49e2-90a2-75ed7200108d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:40.957989 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:56:40.994440 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1356/2742] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:56:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:56:41.006741 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3930cf-9e7c-496e-a46e-54bc8d857101 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:41.008966 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-8b3930cf-9e7c-496e-a46e-54bc8d857101 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:41.097936 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3930cf-9e7c-496e-a46e-54bc8d857101 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:41.101863 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e56cd2a-3b90-4e18-8099-d34ff7a1b059 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:41.103149 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56cd2a-3b90-4e18-8099-d34ff7a1b059 service cinder] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:41.103430 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56cd2a-3b90-4e18-8099-d34ff7a1b059 service cinder] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:41.103790 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56cd2a-3b90-4e18-8099-d34ff7a1b059 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:41.114205 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e56cd2a-3b90-4e18-8099-d34ff7a1b059 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:41.115651 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e56cd2a-3b90-4e18-8099-d34ff7a1b059 service cinder] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:41.125486 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1387/2743] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:56:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:56:41.145949 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc7c76-6e46-49e2-90a2-75ed7200108d None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:41.211111 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc7c76-6e46-49e2-90a2-75ed7200108d None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:41.218697 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d0bbd-a28f-461c-bfa0-e81bb8cb7217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:41.219340 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d0bbd-a28f-461c-bfa0-e81bb8cb7217 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:41.219620 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d0bbd-a28f-461c-bfa0-e81bb8cb7217 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:41.220023 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d0bbd-a28f-461c-bfa0-e81bb8cb7217 admin admin] PATH_INFO: `/v3/users/5a5e267509c948fd9fadb3845e1b984a` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:41.221066 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405d0bbd-a28f-461c-bfa0-e81bb8cb7217 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a5e267509c948fd9fadb3845e1b984a)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:41.225645 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405d0bbd-a28f-461c-bfa0-e81bb8cb7217 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:41.477340 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-405d0bbd-a28f-461c-bfa0-e81bb8cb7217 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a5e267509c948fd9fadb3845e1b984a'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:41.490581 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-405d0bbd-a28f-461c-bfa0-e81bb8cb7217 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a5e267509c948fd9fadb3845e1b984a'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:41.498825 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-405d0bbd-a28f-461c-bfa0-e81bb8cb7217 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a5e267509c948fd9fadb3845e1b984a'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:41.590332 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-405d0bbd-a28f-461c-bfa0-e81bb8cb7217 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a5e267509c948fd9fadb3845e1b984a'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:41.591021 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-405d0bbd-a28f-461c-bfa0-e81bb8cb7217 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a5e267509c948fd9fadb3845e1b984a'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:41.597916 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1357/2744] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:56:41 2022] DELETE /identity/v3/users/5a5e267509c948fd9fadb3845e1b984a => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:56:41.602854 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb46343-cb6c-4648-9059-d678a9606b72 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:41.603256 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb46343-cb6c-4648-9059-d678a9606b72 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:41.603535 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb46343-cb6c-4648-9059-d678a9606b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:41.625352 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-1fb46343-cb6c-4648-9059-d678a9606b72 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:56:41.626490 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-1fb46343-cb6c-4648-9059-d678a9606b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:41.698696 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1388/2745] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:56:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:56:41.710040 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d0bbd-a28f-461c-bfa0-e81bb8cb7217 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:41.712538 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-405d0bbd-a28f-461c-bfa0-e81bb8cb7217 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:41.787987 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d0bbd-a28f-461c-bfa0-e81bb8cb7217 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:41.792449 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a1ab50-9fe7-415a-bf5e-e53a68bfb266 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:41.793289 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1ab50-9fe7-415a-bf5e-e53a68bfb266 service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:41.793674 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1ab50-9fe7-415a-bf5e-e53a68bfb266 service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:41.794077 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1ab50-9fe7-415a-bf5e-e53a68bfb266 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:41.803620 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a1ab50-9fe7-415a-bf5e-e53a68bfb266 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:41.805895 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a1ab50-9fe7-415a-bf5e-e53a68bfb266 service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:41.809659 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:56:41.815958 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1358/2746] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:56:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:56:42.279045 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb46343-cb6c-4648-9059-d678a9606b72 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:42.280948 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-1fb46343-cb6c-4648-9059-d678a9606b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:42.296551 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb46343-cb6c-4648-9059-d678a9606b72 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:42.301776 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e59953-042d-4b55-a9f4-4b2709dc7022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:42.302460 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e59953-042d-4b55-a9f4-4b2709dc7022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:42.302821 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e59953-042d-4b55-a9f4-4b2709dc7022 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:42.303105 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e59953-042d-4b55-a9f4-4b2709dc7022 admin admin] PATH_INFO: `/v3/projects/e1accf0779de46c78b1e93a180b479e9` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:42.304325 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e59953-042d-4b55-a9f4-4b2709dc7022 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1accf0779de46c78b1e93a180b479e9)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:42.307228 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e59953-042d-4b55-a9f4-4b2709dc7022 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:42.327354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:56:42.451789 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-06e59953-042d-4b55-a9f4-4b2709dc7022 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1accf0779de46c78b1e93a180b479e9', 'request_id': 'req-06e59953-042d-4b55-a9f4-4b2709dc7022'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:42.458099 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-06e59953-042d-4b55-a9f4-4b2709dc7022 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1accf0779de46c78b1e93a180b479e9', 'request_id': 'req-06e59953-042d-4b55-a9f4-4b2709dc7022'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:42.465714 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-06e59953-042d-4b55-a9f4-4b2709dc7022 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1accf0779de46c78b1e93a180b479e9', 'request_id': 'req-06e59953-042d-4b55-a9f4-4b2709dc7022'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:42.609683 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-06e59953-042d-4b55-a9f4-4b2709dc7022 admin admin] The token cache is being invalidate because project e1accf0779de46c78b1e93a180b479e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:56:42.610090 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-06e59953-042d-4b55-a9f4-4b2709dc7022 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:42.610774 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-06e59953-042d-4b55-a9f4-4b2709dc7022 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:42.613226 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1389/2747] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:56:42 2022] DELETE /identity/v3/projects/e1accf0779de46c78b1e93a180b479e9 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:56:42.620244 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ee11c-31d6-40f6-a943-07edae960d20 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:42.620911 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ee11c-31d6-40f6-a943-07edae960d20 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:42.621262 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ee11c-31d6-40f6-a943-07edae960d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:42.654259 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-d03ee11c-31d6-40f6-a943-07edae960d20 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:56:42.655764 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-d03ee11c-31d6-40f6-a943-07edae960d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:42.771198 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1359/2748] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:56:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:56:42.780007 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f5ff0-a537-4773-8dc1-56ebdb75d67d None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:42.780223 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f5ff0-a537-4773-8dc1-56ebdb75d67d None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:42.780542 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f5ff0-a537-4773-8dc1-56ebdb75d67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:42.801698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-fb1f5ff0-a537-4773-8dc1-56ebdb75d67d None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:56:42.803075 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-fb1f5ff0-a537-4773-8dc1-56ebdb75d67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:42.813672 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1390/2749] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:56:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:56:42.818119 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03ee11c-31d6-40f6-a943-07edae960d20 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:42.828840 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03ee11c-31d6-40f6-a943-07edae960d20 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:42.833573 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d98c47e-ebc6-434d-9c81-ebfa1c570a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:42.834207 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98c47e-ebc6-434d-9c81-ebfa1c570a50 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:42.834487 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98c47e-ebc6-434d-9c81-ebfa1c570a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:42.834770 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98c47e-ebc6-434d-9c81-ebfa1c570a50 admin admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:42.835434 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d98c47e-ebc6-434d-9c81-ebfa1c570a50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:56:42.835704 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d98c47e-ebc6-434d-9c81-ebfa1c570a50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:42.838532 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d98c47e-ebc6-434d-9c81-ebfa1c570a50 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:42.854089 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1360/2750] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:56:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:56:42.858896 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f5ff0-a537-4773-8dc1-56ebdb75d67d None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:42.869032 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f5ff0-a537-4773-8dc1-56ebdb75d67d None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:42.875686 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a7edd7-db8f-4727-9709-3f6883cfd856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:42.876359 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a7edd7-db8f-4727-9709-3f6883cfd856 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:42.876642 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a7edd7-db8f-4727-9709-3f6883cfd856 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:42.876964 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a7edd7-db8f-4727-9709-3f6883cfd856 admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:42.877604 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a7edd7-db8f-4727-9709-3f6883cfd856 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:42.879633 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a7edd7-db8f-4727-9709-3f6883cfd856 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:42.987097 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1391/2751] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:56:42 2022] POST /identity/v3/projects => generated 383 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:56:42.993257 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d98c47e-ebc6-434d-9c81-ebfa1c570a50 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:43.003623 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d98c47e-ebc6-434d-9c81-ebfa1c570a50 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:43.061738 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1226324a-3c1f-415d-b121-8a2fd2bf6576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:43.062344 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226324a-3c1f-415d-b121-8a2fd2bf6576 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:43.062624 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226324a-3c1f-415d-b121-8a2fd2bf6576 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:43.062897 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226324a-3c1f-415d-b121-8a2fd2bf6576 admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:43.063526 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1226324a-3c1f-415d-b121-8a2fd2bf6576 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:43.065484 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1226324a-3c1f-415d-b121-8a2fd2bf6576 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:43.166614 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1361/2752] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:56:42 2022] POST /identity/v3/users => generated 292 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:56:43.170989 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a7edd7-db8f-4727-9709-3f6883cfd856 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:43.182358 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a7edd7-db8f-4727-9709-3f6883cfd856 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:43.187183 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eca3eb-9588-4722-bf20-b1f8eb0a5a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:43.187858 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eca3eb-9588-4722-bf20-b1f8eb0a5a4f admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:43.188141 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eca3eb-9588-4722-bf20-b1f8eb0a5a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:43.188416 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eca3eb-9588-4722-bf20-b1f8eb0a5a4f admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:43.189096 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eca3eb-9588-4722-bf20-b1f8eb0a5a4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:43.190828 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eca3eb-9588-4722-bf20-b1f8eb0a5a4f admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:43.202443 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1392/2753] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:56:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:56:43.210255 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1226324a-3c1f-415d-b121-8a2fd2bf6576 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:43.219479 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1226324a-3c1f-415d-b121-8a2fd2bf6576 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:43.224744 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e99ff7-501b-465d-b32d-351ded3e4228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:43.225369 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e99ff7-501b-465d-b32d-351ded3e4228 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:43.225709 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e99ff7-501b-465d-b32d-351ded3e4228 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:43.225988 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e99ff7-501b-465d-b32d-351ded3e4228 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:43.226641 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e99ff7-501b-465d-b32d-351ded3e4228 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:43.229185 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e99ff7-501b-465d-b32d-351ded3e4228 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:43.240811 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1362/2754] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:56:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:56:43.246733 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eca3eb-9588-4722-bf20-b1f8eb0a5a4f admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:43.257305 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eca3eb-9588-4722-bf20-b1f8eb0a5a4f admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:43.262639 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d3c43-d1f0-4cc8-a971-18b71dc9db4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:43.263333 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d3c43-d1f0-4cc8-a971-18b71dc9db4d admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:43.263620 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d3c43-d1f0-4cc8-a971-18b71dc9db4d admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:43.263903 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d3c43-d1f0-4cc8-a971-18b71dc9db4d admin admin] PATH_INFO: `/v3/projects/14ebe1294b984d3ab778a9662bad533e/users/fae15e9ba0af4a728d9627c85eb8c33f/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:43.279853 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d3c43-d1f0-4cc8-a971-18b71dc9db4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=14ebe1294b984d3ab778a9662bad533e, user_id=fae15e9ba0af4a728d9627c85eb8c33f, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:43.282671 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d3c43-d1f0-4cc8-a971-18b71dc9db4d admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:43.373992 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1393/2755] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:56:43 2022] PUT /identity/v3/projects/14ebe1294b984d3ab778a9662bad533e/users/fae15e9ba0af4a728d9627c85eb8c33f/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:56:43.386976 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61318ebf-b2af-4569-b5d0-3eefb7d367cc None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:43.387268 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61318ebf-b2af-4569-b5d0-3eefb7d367cc None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:43.387623 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61318ebf-b2af-4569-b5d0-3eefb7d367cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:43.427909 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-61318ebf-b2af-4569-b5d0-3eefb7d367cc None None] MFA Rules not processed for user `fae15e9ba0af4a728d9627c85eb8c33f`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:56:43.429281 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-61318ebf-b2af-4569-b5d0-3eefb7d367cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:43.485327 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:56:43.496782 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1363/2756] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:56:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:56:43.508666 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d3c43-d1f0-4cc8-a971-18b71dc9db4d admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:43.511215 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-5c5d3c43-d1f0-4cc8-a971-18b71dc9db4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:43.592175 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d3c43-d1f0-4cc8-a971-18b71dc9db4d admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:43.596084 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ddc9b6-23ad-4598-86d3-a26dd6c7c14e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:43.596705 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddc9b6-23ad-4598-86d3-a26dd6c7c14e service cinder] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:43.597006 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddc9b6-23ad-4598-86d3-a26dd6c7c14e service cinder] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:43.597280 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddc9b6-23ad-4598-86d3-a26dd6c7c14e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:43.606595 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ddc9b6-23ad-4598-86d3-a26dd6c7c14e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:43.615360 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ddc9b6-23ad-4598-86d3-a26dd6c7c14e service cinder] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:43.618335 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1394/2757] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:56:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:56:46.787737 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61318ebf-b2af-4569-b5d0-3eefb7d367cc None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:46.853575 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61318ebf-b2af-4569-b5d0-3eefb7d367cc None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:46.858262 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e46b4a-821b-4548-8e00-f532f7fae91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:46.858892 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e46b4a-821b-4548-8e00-f532f7fae91e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:46.861963 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e46b4a-821b-4548-8e00-f532f7fae91e admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:46.861963 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e46b4a-821b-4548-8e00-f532f7fae91e admin admin] PATH_INFO: `/v3/users/fae15e9ba0af4a728d9627c85eb8c33f` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:46.861963 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e46b4a-821b-4548-8e00-f532f7fae91e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fae15e9ba0af4a728d9627c85eb8c33f)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:46.862752 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e46b4a-821b-4548-8e00-f532f7fae91e admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:47.102964 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-37e46b4a-821b-4548-8e00-f532f7fae91e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fae15e9ba0af4a728d9627c85eb8c33f'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:47.117929 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-37e46b4a-821b-4548-8e00-f532f7fae91e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fae15e9ba0af4a728d9627c85eb8c33f'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:47.125075 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-37e46b4a-821b-4548-8e00-f532f7fae91e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fae15e9ba0af4a728d9627c85eb8c33f'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:47.200984 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-37e46b4a-821b-4548-8e00-f532f7fae91e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fae15e9ba0af4a728d9627c85eb8c33f'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:47.201616 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-37e46b4a-821b-4548-8e00-f532f7fae91e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fae15e9ba0af4a728d9627c85eb8c33f'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:47.209043 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1364/2758] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:56:46 2022] DELETE /identity/v3/users/fae15e9ba0af4a728d9627c85eb8c33f => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:56:47.214208 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d37f58-dcf6-4d74-a720-54d675d97063 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:47.214732 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d37f58-dcf6-4d74-a720-54d675d97063 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:47.214857 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d37f58-dcf6-4d74-a720-54d675d97063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:47.237863 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-65d37f58-dcf6-4d74-a720-54d675d97063 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:56:47.239139 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-65d37f58-dcf6-4d74-a720-54d675d97063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:47.282421 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:56:47.301089 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1395/2759] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:56:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:56:47.311033 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e46b4a-821b-4548-8e00-f532f7fae91e admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:47.313311 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-37e46b4a-821b-4548-8e00-f532f7fae91e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:47.372393 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e46b4a-821b-4548-8e00-f532f7fae91e admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:47.376028 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5fce5-3514-4572-9401-7dfb93b60d11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:47.376636 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5fce5-3514-4572-9401-7dfb93b60d11 service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:47.376959 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5fce5-3514-4572-9401-7dfb93b60d11 service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:47.377239 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5fce5-3514-4572-9401-7dfb93b60d11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:47.387118 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f5fce5-3514-4572-9401-7dfb93b60d11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:47.388552 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f5fce5-3514-4572-9401-7dfb93b60d11 service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:47.398164 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1365/2760] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:56:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:56:47.923901 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d37f58-dcf6-4d74-a720-54d675d97063 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:47.925372 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-65d37f58-dcf6-4d74-a720-54d675d97063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:47.938559 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d37f58-dcf6-4d74-a720-54d675d97063 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:47.943863 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50baac0f-ab6d-4756-920f-4ef801128df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:47.944560 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50baac0f-ab6d-4756-920f-4ef801128df5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:47.944975 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50baac0f-ab6d-4756-920f-4ef801128df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:47.945315 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50baac0f-ab6d-4756-920f-4ef801128df5 admin admin] PATH_INFO: `/v3/projects/14ebe1294b984d3ab778a9662bad533e` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:47.946541 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50baac0f-ab6d-4756-920f-4ef801128df5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14ebe1294b984d3ab778a9662bad533e)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:47.949691 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50baac0f-ab6d-4756-920f-4ef801128df5 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:48.064609 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-50baac0f-ab6d-4756-920f-4ef801128df5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14ebe1294b984d3ab778a9662bad533e', 'request_id': 'req-50baac0f-ab6d-4756-920f-4ef801128df5'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:48.070156 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-50baac0f-ab6d-4756-920f-4ef801128df5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14ebe1294b984d3ab778a9662bad533e', 'request_id': 'req-50baac0f-ab6d-4756-920f-4ef801128df5'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:48.077463 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-50baac0f-ab6d-4756-920f-4ef801128df5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14ebe1294b984d3ab778a9662bad533e', 'request_id': 'req-50baac0f-ab6d-4756-920f-4ef801128df5'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:48.172451 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-50baac0f-ab6d-4756-920f-4ef801128df5 admin admin] The token cache is being invalidate because project 14ebe1294b984d3ab778a9662bad533e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:56:48.172652 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-50baac0f-ab6d-4756-920f-4ef801128df5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:48.173299 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-50baac0f-ab6d-4756-920f-4ef801128df5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:48.174772 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1396/2761] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:56:47 2022] DELETE /identity/v3/projects/14ebe1294b984d3ab778a9662bad533e => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:56:48.179824 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420fd7d-44bd-49ad-9dee-b77aa6136ccf None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:48.180286 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420fd7d-44bd-49ad-9dee-b77aa6136ccf None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:48.180385 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420fd7d-44bd-49ad-9dee-b77aa6136ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:48.202494 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-4420fd7d-44bd-49ad-9dee-b77aa6136ccf None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:56:48.203679 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-4420fd7d-44bd-49ad-9dee-b77aa6136ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:48.213391 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:56:48.267747 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1366/2762] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:56:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:56:48.278982 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d986a-871a-413a-90fa-24aa930b0c32 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:48.279367 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d986a-871a-413a-90fa-24aa930b0c32 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:48.279472 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d986a-871a-413a-90fa-24aa930b0c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:48.301004 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-339d986a-871a-413a-90fa-24aa930b0c32 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:56:48.302269 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-339d986a-871a-413a-90fa-24aa930b0c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:48.311301 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1397/2763] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:56:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:56:48.315666 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420fd7d-44bd-49ad-9dee-b77aa6136ccf None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:48.328226 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420fd7d-44bd-49ad-9dee-b77aa6136ccf None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:48.333136 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a14a0-4b13-4d14-8963-8c1368c73d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:48.333749 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a14a0-4b13-4d14-8963-8c1368c73d6a admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:48.334031 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a14a0-4b13-4d14-8963-8c1368c73d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:48.334311 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a14a0-4b13-4d14-8963-8c1368c73d6a admin admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:48.334962 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0a14a0-4b13-4d14-8963-8c1368c73d6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:56:48.335266 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0a14a0-4b13-4d14-8963-8c1368c73d6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:48.337253 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0a14a0-4b13-4d14-8963-8c1368c73d6a admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:48.353797 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1367/2764] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 22:56:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:56:48.355059 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339d986a-871a-413a-90fa-24aa930b0c32 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:48.366356 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339d986a-871a-413a-90fa-24aa930b0c32 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:48.372660 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21efcc6-82b5-4fd4-b303-c084bbf4e64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:48.373435 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21efcc6-82b5-4fd4-b303-c084bbf4e64f admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:48.373721 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21efcc6-82b5-4fd4-b303-c084bbf4e64f admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:48.373997 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21efcc6-82b5-4fd4-b303-c084bbf4e64f admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:48.374663 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21efcc6-82b5-4fd4-b303-c084bbf4e64f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:48.376589 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21efcc6-82b5-4fd4-b303-c084bbf4e64f admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:48.444499 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1398/2765] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 22:56:48 2022] POST /identity/v3/projects => generated 371 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:56:48.449758 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a14a0-4b13-4d14-8963-8c1368c73d6a admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:48.462436 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a14a0-4b13-4d14-8963-8c1368c73d6a admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:48.520015 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e4f01b-983e-4d34-bcb1-02570b1c2a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:48.520617 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e4f01b-983e-4d34-bcb1-02570b1c2a1f admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:48.520929 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e4f01b-983e-4d34-bcb1-02570b1c2a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:48.521200 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e4f01b-983e-4d34-bcb1-02570b1c2a1f admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:48.521842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e4f01b-983e-4d34-bcb1-02570b1c2a1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:48.523767 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e4f01b-983e-4d34-bcb1-02570b1c2a1f admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:48.641343 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1368/2766] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 22:56:48 2022] POST /identity/v3/users => generated 286 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:56:48.645570 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21efcc6-82b5-4fd4-b303-c084bbf4e64f admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:48.656203 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21efcc6-82b5-4fd4-b303-c084bbf4e64f admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:48.660887 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd79b7-018d-4fb1-8eae-6c376600b218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:48.661481 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd79b7-018d-4fb1-8eae-6c376600b218 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:48.661758 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd79b7-018d-4fb1-8eae-6c376600b218 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:48.662029 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd79b7-018d-4fb1-8eae-6c376600b218 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:48.662669 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd79b7-018d-4fb1-8eae-6c376600b218 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:48.664294 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd79b7-018d-4fb1-8eae-6c376600b218 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:48.675048 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1399/2767] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:56:48.682102 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e4f01b-983e-4d34-bcb1-02570b1c2a1f admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:48.692238 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e4f01b-983e-4d34-bcb1-02570b1c2a1f admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:48.696943 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e57058-74fc-43ae-b334-53a09b33a55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:48.697545 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e57058-74fc-43ae-b334-53a09b33a55b admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:48.697828 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e57058-74fc-43ae-b334-53a09b33a55b admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:48.698550 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e57058-74fc-43ae-b334-53a09b33a55b admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:48.698758 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e57058-74fc-43ae-b334-53a09b33a55b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:48.700389 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e57058-74fc-43ae-b334-53a09b33a55b admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:48.713214 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1369/2768] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 22:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:56:48.717461 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd79b7-018d-4fb1-8eae-6c376600b218 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:48.727543 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd79b7-018d-4fb1-8eae-6c376600b218 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:48.733001 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d70ee4-76ed-4449-9c99-5011ede2c7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:48.733648 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d70ee4-76ed-4449-9c99-5011ede2c7bc admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:48.733926 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d70ee4-76ed-4449-9c99-5011ede2c7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:48.734197 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d70ee4-76ed-4449-9c99-5011ede2c7bc admin admin] PATH_INFO: `/v3/projects/f70a40e5324a45cf87b0eb597e774348/users/132d21d3bc524a48bbe9afdc1d484d01/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:48.749389 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d70ee4-76ed-4449-9c99-5011ede2c7bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f70a40e5324a45cf87b0eb597e774348, user_id=132d21d3bc524a48bbe9afdc1d484d01, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:48.752119 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d70ee4-76ed-4449-9c99-5011ede2c7bc admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:48.829784 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1400/2769] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 22:56:48 2022] PUT /identity/v3/projects/f70a40e5324a45cf87b0eb597e774348/users/132d21d3bc524a48bbe9afdc1d484d01/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:56:48.842486 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73528813-058c-4d4f-b82e-47a4c09a02ec None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:48.842774 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73528813-058c-4d4f-b82e-47a4c09a02ec None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:48.843053 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73528813-058c-4d4f-b82e-47a4c09a02ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:48.873232 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-73528813-058c-4d4f-b82e-47a4c09a02ec None None] MFA Rules not processed for user `132d21d3bc524a48bbe9afdc1d484d01`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:56:48.874448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-73528813-058c-4d4f-b82e-47a4c09a02ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:48.942056 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1370/2770] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:56:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:56:48.954405 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d70ee4-76ed-4449-9c99-5011ede2c7bc admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:48.956660 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-e5d70ee4-76ed-4449-9c99-5011ede2c7bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:49.015860 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d70ee4-76ed-4449-9c99-5011ede2c7bc admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:49.019839 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74659b1a-4d52-418d-9283-bdff39f39b03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:49.020455 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74659b1a-4d52-418d-9283-bdff39f39b03 service cinder] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:49.020944 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74659b1a-4d52-418d-9283-bdff39f39b03 service cinder] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:49.021178 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74659b1a-4d52-418d-9283-bdff39f39b03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:49.030990 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74659b1a-4d52-418d-9283-bdff39f39b03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:49.032915 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74659b1a-4d52-418d-9283-bdff39f39b03 service cinder] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:49.043372 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1401/2771] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:56:48 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:56:49.069103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73528813-058c-4d4f-b82e-47a4c09a02ec None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:49.081562 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:56:49.131748 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73528813-058c-4d4f-b82e-47a4c09a02ec None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:49.136542 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e94ae4-9213-4655-84d6-31dcb91efaee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:49.137206 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e94ae4-9213-4655-84d6-31dcb91efaee admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:49.137493 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e94ae4-9213-4655-84d6-31dcb91efaee admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:49.137773 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e94ae4-9213-4655-84d6-31dcb91efaee admin admin] PATH_INFO: `/v3/users/132d21d3bc524a48bbe9afdc1d484d01` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:49.138948 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e94ae4-9213-4655-84d6-31dcb91efaee admin admin] RBAC: Authorizing `identity:delete_user(user_id=132d21d3bc524a48bbe9afdc1d484d01)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:49.140895 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e94ae4-9213-4655-84d6-31dcb91efaee admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:49.354052 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-a1e94ae4-9213-4655-84d6-31dcb91efaee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '132d21d3bc524a48bbe9afdc1d484d01'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:49.366247 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-a1e94ae4-9213-4655-84d6-31dcb91efaee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '132d21d3bc524a48bbe9afdc1d484d01'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:49.373768 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-a1e94ae4-9213-4655-84d6-31dcb91efaee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '132d21d3bc524a48bbe9afdc1d484d01'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:49.453265 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-a1e94ae4-9213-4655-84d6-31dcb91efaee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '132d21d3bc524a48bbe9afdc1d484d01'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:49.454013 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-a1e94ae4-9213-4655-84d6-31dcb91efaee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '132d21d3bc524a48bbe9afdc1d484d01'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:49.464042 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1371/2772] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:56:49 2022] DELETE /identity/v3/users/132d21d3bc524a48bbe9afdc1d484d01 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:56:49.468202 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713cd99-418d-4a66-a434-8d43810d3881 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:49.468785 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713cd99-418d-4a66-a434-8d43810d3881 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:49.468785 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713cd99-418d-4a66-a434-8d43810d3881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:49.490359 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:56:49.495228 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-e713cd99-418d-4a66-a434-8d43810d3881 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:56:49.496658 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-e713cd99-418d-4a66-a434-8d43810d3881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:49.573268 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1402/2773] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:56:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:56:49.582851 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e94ae4-9213-4655-84d6-31dcb91efaee admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:49.584303 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-a1e94ae4-9213-4655-84d6-31dcb91efaee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:49.646637 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e94ae4-9213-4655-84d6-31dcb91efaee admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:49.650719 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af40800f-767e-4ab7-9263-6f95ca23cf9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:49.651445 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af40800f-767e-4ab7-9263-6f95ca23cf9f service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:49.656075 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af40800f-767e-4ab7-9263-6f95ca23cf9f service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:49.656075 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af40800f-767e-4ab7-9263-6f95ca23cf9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:49.661673 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af40800f-767e-4ab7-9263-6f95ca23cf9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:49.663173 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af40800f-767e-4ab7-9263-6f95ca23cf9f service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:49.672982 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1372/2774] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:56:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:56:50.151668 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e713cd99-418d-4a66-a434-8d43810d3881 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:50.153235 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-e713cd99-418d-4a66-a434-8d43810d3881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:56:50.167876 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e713cd99-418d-4a66-a434-8d43810d3881 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:50.172594 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddca52c9-4483-4800-bf9f-8d7d6476fd6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:50.173265 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddca52c9-4483-4800-bf9f-8d7d6476fd6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:50.173544 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddca52c9-4483-4800-bf9f-8d7d6476fd6b admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:50.173816 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddca52c9-4483-4800-bf9f-8d7d6476fd6b admin admin] PATH_INFO: `/v3/projects/f70a40e5324a45cf87b0eb597e774348` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:50.175079 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddca52c9-4483-4800-bf9f-8d7d6476fd6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f70a40e5324a45cf87b0eb597e774348)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:50.177077 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddca52c9-4483-4800-bf9f-8d7d6476fd6b admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:50.300183 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ddca52c9-4483-4800-bf9f-8d7d6476fd6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f70a40e5324a45cf87b0eb597e774348', 'request_id': 'req-ddca52c9-4483-4800-bf9f-8d7d6476fd6b'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:50.306674 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ddca52c9-4483-4800-bf9f-8d7d6476fd6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f70a40e5324a45cf87b0eb597e774348', 'request_id': 'req-ddca52c9-4483-4800-bf9f-8d7d6476fd6b'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:50.313210 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ddca52c9-4483-4800-bf9f-8d7d6476fd6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f70a40e5324a45cf87b0eb597e774348', 'request_id': 'req-ddca52c9-4483-4800-bf9f-8d7d6476fd6b'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:50.389493 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ddca52c9-4483-4800-bf9f-8d7d6476fd6b admin admin] The token cache is being invalidate because project f70a40e5324a45cf87b0eb597e774348 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:56:50.389780 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ddca52c9-4483-4800-bf9f-8d7d6476fd6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:50.390417 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ddca52c9-4483-4800-bf9f-8d7d6476fd6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:56:50.391906 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1403/2775] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:56:50 2022] DELETE /identity/v3/projects/f70a40e5324a45cf87b0eb597e774348 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:56:59.668740 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=74,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:57:00.386231 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:57:46.462789 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ebaa5-cfed-4f67-914e-0bbf3b46a20e None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:57:46.463323 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ebaa5-cfed-4f67-914e-0bbf3b46a20e None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:57:46.463323 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ebaa5-cfed-4f67-914e-0bbf3b46a20e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:57:46.482722 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-ca2ebaa5-cfed-4f67-914e-0bbf3b46a20e None None] MFA Rules not processed for user `f3121e61f02e4869b8214a5caf1b580a`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:57:46.484537 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-ca2ebaa5-cfed-4f67-914e-0bbf3b46a20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:57:46.541702 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1373/2776] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:57:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:57:56.534438 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=25 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:58:16.809603 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddca52c9-4483-4800-bf9f-8d7d6476fd6b admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:58:16.811143 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-ddca52c9-4483-4800-bf9f-8d7d6476fd6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:58:16.876219 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddca52c9-4483-4800-bf9f-8d7d6476fd6b admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:58:16.879989 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6479a3f-2a50-4078-b460-0210b4084aa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:58:16.880604 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6479a3f-2a50-4078-b460-0210b4084aa9 service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:58:16.880918 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6479a3f-2a50-4078-b460-0210b4084aa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:58:16.881198 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6479a3f-2a50-4078-b460-0210b4084aa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:58:16.882993 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-a6479a3f-2a50-4078-b460-0210b4084aa9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:58:16.980285 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6479a3f-2a50-4078-b460-0210b4084aa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:58:16.981830 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6479a3f-2a50-4078-b460-0210b4084aa9 service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:58:17.006770 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1404/2777] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:58:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:58:18.107883 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d410d024-2a25-405f-9733-79e6bbe2dc16 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:58:18.108379 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d410d024-2a25-405f-9733-79e6bbe2dc16 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:58:18.108484 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d410d024-2a25-405f-9733-79e6bbe2dc16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:58:18.129938 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-d410d024-2a25-405f-9733-79e6bbe2dc16 None None] MFA Rules not processed for user `df23a24f4d5948deb641de1cefad1f69`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:58:18.131413 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-d410d024-2a25-405f-9733-79e6bbe2dc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:58:18.186597 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1374/2778] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:58:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:58:19.235002 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6479a3f-2a50-4078-b460-0210b4084aa9 service neutron] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:58:19.236656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-a6479a3f-2a50-4078-b460-0210b4084aa9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:58:19.247436 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6479a3f-2a50-4078-b460-0210b4084aa9 service neutron] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:58:19.251555 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676eab6-8e3e-4cbb-8850-b7c42ac2fbd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:58:19.252163 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676eab6-8e3e-4cbb-8850-b7c42ac2fbd0 service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:58:19.252442 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676eab6-8e3e-4cbb-8850-b7c42ac2fbd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:58:19.252799 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676eab6-8e3e-4cbb-8850-b7c42ac2fbd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:58:19.254492 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-2676eab6-8e3e-4cbb-8850-b7c42ac2fbd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:58:19.269428 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2676eab6-8e3e-4cbb-8850-b7c42ac2fbd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:58:19.270862 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2676eab6-8e3e-4cbb-8850-b7c42ac2fbd0 service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:58:19.282073 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1405/2779] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:58:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:58:28.179912 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=25 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:58:29.277199 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=72 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:58:47.490991 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d410d024-2a25-405f-9733-79e6bbe2dc16 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:58:47.503996 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d410d024-2a25-405f-9733-79e6bbe2dc16 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:58:47.507821 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a7fa9-dfce-4ed4-99da-cfd660d0ab59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:58:47.508435 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a7fa9-dfce-4ed4-99da-cfd660d0ab59 service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:58:47.508737 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a7fa9-dfce-4ed4-99da-cfd660d0ab59 service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:58:47.509025 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a7fa9-dfce-4ed4-99da-cfd660d0ab59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:58:47.510781 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-755a7fa9-dfce-4ed4-99da-cfd660d0ab59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:58:47.521413 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a7fa9-dfce-4ed4-99da-cfd660d0ab59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:58:47.523074 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a7fa9-dfce-4ed4-99da-cfd660d0ab59 service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:58:47.532245 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1375/2780] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:58:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:58:54.091064 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676eab6-8e3e-4cbb-8850-b7c42ac2fbd0 service neutron] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:58:54.100742 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676eab6-8e3e-4cbb-8850-b7c42ac2fbd0 service neutron] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:58:54.104541 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe53c426-275e-4abc-99e8-6d0d201c06f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:58:54.105178 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe53c426-275e-4abc-99e8-6d0d201c06f5 service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:58:54.105460 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe53c426-275e-4abc-99e8-6d0d201c06f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:58:54.105804 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe53c426-275e-4abc-99e8-6d0d201c06f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:58:54.107649 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-fe53c426-275e-4abc-99e8-6d0d201c06f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:58:54.179133 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe53c426-275e-4abc-99e8-6d0d201c06f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:58:54.180930 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe53c426-275e-4abc-99e8-6d0d201c06f5 service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:58:54.192089 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1406/2781] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:58:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:58:57.527709 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=6 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:58:59.470199 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe1374-2280-45a2-921b-fa6b69a522a5 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:58:59.472283 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe1374-2280-45a2-921b-fa6b69a522a5 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:58:59.472283 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe1374-2280-45a2-921b-fa6b69a522a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:58:59.492343 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-d6fe1374-2280-45a2-921b-fa6b69a522a5 None None] MFA Rules not processed for user `f3121e61f02e4869b8214a5caf1b580a`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:58:59.493603 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-d6fe1374-2280-45a2-921b-fa6b69a522a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:58:59.501278 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1376/2782] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 22:58:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:58:59.825931 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe53c426-275e-4abc-99e8-6d0d201c06f5 service neutron] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:58:59.837080 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe53c426-275e-4abc-99e8-6d0d201c06f5 service neutron] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:58:59.841603 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ddfc8-2e28-4991-8d3d-392728361076 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:58:59.842225 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ddfc8-2e28-4991-8d3d-392728361076 service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:58:59.842530 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ddfc8-2e28-4991-8d3d-392728361076 service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:58:59.842803 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ddfc8-2e28-4991-8d3d-392728361076 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:58:59.845314 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-6a1ddfc8-2e28-4991-8d3d-392728361076 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:58:59.856506 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1ddfc8-2e28-4991-8d3d-392728361076 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:58:59.858014 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1ddfc8-2e28-4991-8d3d-392728361076 service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:58:59.867205 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1407/2783] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:58:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:59:00.293614 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe1374-2280-45a2-921b-fa6b69a522a5 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:59:00.294696 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-d6fe1374-2280-45a2-921b-fa6b69a522a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:59:00.401410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe1374-2280-45a2-921b-fa6b69a522a5 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:59:00.401410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1dbf44-4298-4899-b4bb-da272894a824 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8fd0f6e5b144d2ca78b8d96722502a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:59:00.401410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1dbf44-4298-4899-b4bb-da272894a824 service placement] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:59:00.401410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1dbf44-4298-4899-b4bb-da272894a824 service placement] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:59:00.402051 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1dbf44-4298-4899-b4bb-da272894a824 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:59:00.402811 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-5e1dbf44-4298-4899-b4bb-da272894a824 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:59:00.416579 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1dbf44-4298-4899-b4bb-da272894a824 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:59:00.418273 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1dbf44-4298-4899-b4bb-da272894a824 service placement] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:59:00.429332 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1377/2784] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:59:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:59:02.007845 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ddfc8-2e28-4991-8d3d-392728361076 service neutron] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:59:02.009341 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-6a1ddfc8-2e28-4991-8d3d-392728361076 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:59:02.026078 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ddfc8-2e28-4991-8d3d-392728361076 service neutron] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:59:02.030928 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a24adcc-3bf2-4f9a-b9e2-71fdc9895214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:59:02.031870 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a24adcc-3bf2-4f9a-b9e2-71fdc9895214 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:59:02.032173 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a24adcc-3bf2-4f9a-b9e2-71fdc9895214 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:59:02.032427 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a24adcc-3bf2-4f9a-b9e2-71fdc9895214 admin admin] PATH_INFO: `/v3/users/5f04bf8fdc2c4451a3cb6a9929273973` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:59:02.033638 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a24adcc-3bf2-4f9a-b9e2-71fdc9895214 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f04bf8fdc2c4451a3cb6a9929273973)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:59:02.035701 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a24adcc-3bf2-4f9a-b9e2-71fdc9895214 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:59:02.360920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-3a24adcc-3bf2-4f9a-b9e2-71fdc9895214 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f04bf8fdc2c4451a3cb6a9929273973'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:59:02.429305 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-3a24adcc-3bf2-4f9a-b9e2-71fdc9895214 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f04bf8fdc2c4451a3cb6a9929273973'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:59:02.441948 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-3a24adcc-3bf2-4f9a-b9e2-71fdc9895214 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f04bf8fdc2c4451a3cb6a9929273973'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:59:02.449015 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-3a24adcc-3bf2-4f9a-b9e2-71fdc9895214 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f04bf8fdc2c4451a3cb6a9929273973'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:59:02.449715 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-3a24adcc-3bf2-4f9a-b9e2-71fdc9895214 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f04bf8fdc2c4451a3cb6a9929273973'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:59:02.455656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1408/2785] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:59:02 2022] DELETE /identity/v3/users/5f04bf8fdc2c4451a3cb6a9929273973 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:59:02.811538 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1dbf44-4298-4899-b4bb-da272894a824 service placement] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:59:02.822087 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1dbf44-4298-4899-b4bb-da272894a824 service placement] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:59:02.896700 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e853df36-ce14-4eae-a276-e31b01feb485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:59:02.897775 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853df36-ce14-4eae-a276-e31b01feb485 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:59:02.897775 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853df36-ce14-4eae-a276-e31b01feb485 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:59:02.897989 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853df36-ce14-4eae-a276-e31b01feb485 admin admin] PATH_INFO: `/v3/projects/e1e766828cca43808ba55f00a50c53f9` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:59:02.899008 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e853df36-ce14-4eae-a276-e31b01feb485 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1e766828cca43808ba55f00a50c53f9)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:59:02.900985 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e853df36-ce14-4eae-a276-e31b01feb485 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:59:03.051276 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-e853df36-ce14-4eae-a276-e31b01feb485 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1e766828cca43808ba55f00a50c53f9', 'request_id': 'req-e853df36-ce14-4eae-a276-e31b01feb485'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:59:03.156489 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-e853df36-ce14-4eae-a276-e31b01feb485 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1e766828cca43808ba55f00a50c53f9', 'request_id': 'req-e853df36-ce14-4eae-a276-e31b01feb485'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:59:03.164611 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-e853df36-ce14-4eae-a276-e31b01feb485 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1e766828cca43808ba55f00a50c53f9', 'request_id': 'req-e853df36-ce14-4eae-a276-e31b01feb485'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:59:03.178243 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-e853df36-ce14-4eae-a276-e31b01feb485 admin admin] The token cache is being invalidate because project e1e766828cca43808ba55f00a50c53f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:59:03.178531 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-e853df36-ce14-4eae-a276-e31b01feb485 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:59:03.179197 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-e853df36-ce14-4eae-a276-e31b01feb485 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:59:03.181143 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1378/2786] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:59:02 2022] DELETE /identity/v3/projects/e1e766828cca43808ba55f00a50c53f9 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:59:03.185049 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a24adcc-3bf2-4f9a-b9e2-71fdc9895214 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:59:03.186626 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-3a24adcc-3bf2-4f9a-b9e2-71fdc9895214 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:59:03.202711 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a24adcc-3bf2-4f9a-b9e2-71fdc9895214 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:59:03.259441 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:59:03.270102 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5fdf12-3700-4384-a6cc-debdd65c5942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:59:03.270745 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5fdf12-3700-4384-a6cc-debdd65c5942 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:59:03.271024 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5fdf12-3700-4384-a6cc-debdd65c5942 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:59:03.271298 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5fdf12-3700-4384-a6cc-debdd65c5942 admin admin] PATH_INFO: `/v3/users/f04e671081ac4c9db7c389a88fed9b42` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:59:03.272512 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5fdf12-3700-4384-a6cc-debdd65c5942 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f04e671081ac4c9db7c389a88fed9b42)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:59:03.274477 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5fdf12-3700-4384-a6cc-debdd65c5942 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:59:03.468862 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ae5fdf12-3700-4384-a6cc-debdd65c5942 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f04e671081ac4c9db7c389a88fed9b42'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:59:03.558691 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ae5fdf12-3700-4384-a6cc-debdd65c5942 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f04e671081ac4c9db7c389a88fed9b42'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:59:03.575398 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ae5fdf12-3700-4384-a6cc-debdd65c5942 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f04e671081ac4c9db7c389a88fed9b42'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:59:03.582761 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ae5fdf12-3700-4384-a6cc-debdd65c5942 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f04e671081ac4c9db7c389a88fed9b42'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:59:03.583791 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ae5fdf12-3700-4384-a6cc-debdd65c5942 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f04e671081ac4c9db7c389a88fed9b42'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:59:03.593203 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1409/2787] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 22:59:03 2022] DELETE /identity/v3/users/f04e671081ac4c9db7c389a88fed9b42 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:59:03.949079 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e853df36-ce14-4eae-a276-e31b01feb485 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:59:03.951105 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-e853df36-ce14-4eae-a276-e31b01feb485 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:59:03.965136 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e853df36-ce14-4eae-a276-e31b01feb485 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:59:03.976566 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:59:04.025627 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85461d3-2202-4a4d-9177-b3e58516e727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:59:04.026258 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85461d3-2202-4a4d-9177-b3e58516e727 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:59:04.026537 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85461d3-2202-4a4d-9177-b3e58516e727 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:59:04.026806 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85461d3-2202-4a4d-9177-b3e58516e727 admin admin] PATH_INFO: `/v3/projects/4c5041b479ac4dcfa6d5c53793fa9156` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:59:04.027923 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85461d3-2202-4a4d-9177-b3e58516e727 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c5041b479ac4dcfa6d5c53793fa9156)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:59:04.029856 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85461d3-2202-4a4d-9177-b3e58516e727 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:59:04.201214 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-e85461d3-2202-4a4d-9177-b3e58516e727 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c5041b479ac4dcfa6d5c53793fa9156', 'request_id': 'req-e85461d3-2202-4a4d-9177-b3e58516e727'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:59:04.301158 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-e85461d3-2202-4a4d-9177-b3e58516e727 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c5041b479ac4dcfa6d5c53793fa9156', 'request_id': 'req-e85461d3-2202-4a4d-9177-b3e58516e727'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:59:04.307496 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-e85461d3-2202-4a4d-9177-b3e58516e727 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c5041b479ac4dcfa6d5c53793fa9156', 'request_id': 'req-e85461d3-2202-4a4d-9177-b3e58516e727'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:59:04.321120 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-e85461d3-2202-4a4d-9177-b3e58516e727 admin admin] The token cache is being invalidate because project 4c5041b479ac4dcfa6d5c53793fa9156 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:59:04.321474 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-e85461d3-2202-4a4d-9177-b3e58516e727 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:59:04.322076 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-e85461d3-2202-4a4d-9177-b3e58516e727 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:59:04.323798 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1379/2788] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 22:59:03 2022] DELETE /identity/v3/projects/4c5041b479ac4dcfa6d5c53793fa9156 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:59:04.510633 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5fdf12-3700-4384-a6cc-debdd65c5942 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:59:04.512235 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-ae5fdf12-3700-4384-a6cc-debdd65c5942 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:59:04.612445 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5fdf12-3700-4384-a6cc-debdd65c5942 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:59:04.679099 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c9629-c55a-4209-a552-e83385c92015 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:59:04.679713 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c9629-c55a-4209-a552-e83385c92015 service nova] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:59:04.679993 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c9629-c55a-4209-a552-e83385c92015 service nova] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:59:04.680267 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c9629-c55a-4209-a552-e83385c92015 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:59:04.682108 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-3b7c9629-c55a-4209-a552-e83385c92015 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:59:04.693021 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:59:04.695694 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7c9629-c55a-4209-a552-e83385c92015 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:59:04.697352 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7c9629-c55a-4209-a552-e83385c92015 service nova] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:59:04.709478 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1410/2789] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 22:59:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:59:14.418075 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=45,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:59:14.792612 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=2 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:00:17.459798 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f8a50-5ba0-4914-aa40-8fda34de815d None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:17.460362 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f8a50-5ba0-4914-aa40-8fda34de815d None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:17.460362 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f8a50-5ba0-4914-aa40-8fda34de815d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:17.481066 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-001f8a50-5ba0-4914-aa40-8fda34de815d None None] MFA Rules not processed for user `df23a24f4d5948deb641de1cefad1f69`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:00:17.482305 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-001f8a50-5ba0-4914-aa40-8fda34de815d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:17.563213 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1380/2790] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 23:00:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:00:18.000185 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c9629-c55a-4209-a552-e83385c92015 service nova] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:18.001651 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-3b7c9629-c55a-4209-a552-e83385c92015 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:18.083072 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c9629-c55a-4209-a552-e83385c92015 service nova] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:18.088360 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4611a-794a-416d-aac8-0c9b549f6621 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8fd0f6e5b144d2ca78b8d96722502a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:18.089233 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4611a-794a-416d-aac8-0c9b549f6621 service placement] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:18.089600 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4611a-794a-416d-aac8-0c9b549f6621 service placement] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:18.089926 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4611a-794a-416d-aac8-0c9b549f6621 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:18.091818 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-b5c4611a-794a-416d-aac8-0c9b549f6621 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:18.105363 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c4611a-794a-416d-aac8-0c9b549f6621 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:18.107183 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c4611a-794a-416d-aac8-0c9b549f6621 service placement] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:18.118390 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1411/2791] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 23:00:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:00:18.168628 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f8a50-5ba0-4914-aa40-8fda34de815d None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:18.174334 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-001f8a50-5ba0-4914-aa40-8fda34de815d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:18.249050 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f8a50-5ba0-4914-aa40-8fda34de815d None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:18.253855 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6baa090-fb2a-4082-b3ed-765f2476c6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:18.254506 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6baa090-fb2a-4082-b3ed-765f2476c6b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:18.254905 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6baa090-fb2a-4082-b3ed-765f2476c6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:18.255184 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6baa090-fb2a-4082-b3ed-765f2476c6b7 admin admin] PATH_INFO: `/v3/users/f3121e61f02e4869b8214a5caf1b580a` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:18.256386 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6baa090-fb2a-4082-b3ed-765f2476c6b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3121e61f02e4869b8214a5caf1b580a)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:18.258362 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6baa090-fb2a-4082-b3ed-765f2476c6b7 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:18.433391 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-e6baa090-fb2a-4082-b3ed-765f2476c6b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3121e61f02e4869b8214a5caf1b580a'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:18.445225 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-e6baa090-fb2a-4082-b3ed-765f2476c6b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3121e61f02e4869b8214a5caf1b580a'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:18.452675 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-e6baa090-fb2a-4082-b3ed-765f2476c6b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3121e61f02e4869b8214a5caf1b580a'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:18.537998 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-e6baa090-fb2a-4082-b3ed-765f2476c6b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3121e61f02e4869b8214a5caf1b580a'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:18.538648 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-e6baa090-fb2a-4082-b3ed-765f2476c6b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3121e61f02e4869b8214a5caf1b580a'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:18.546876 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1381/2792] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 23:00:18 2022] DELETE /identity/v3/users/f3121e61f02e4869b8214a5caf1b580a => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:00:18.793215 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4611a-794a-416d-aac8-0c9b549f6621 service placement] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:18.812046 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4611a-794a-416d-aac8-0c9b549f6621 service placement] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:18.886753 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198c3f9c-a758-440f-89cf-6d8d26741238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:18.887416 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198c3f9c-a758-440f-89cf-6d8d26741238 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:18.887695 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198c3f9c-a758-440f-89cf-6d8d26741238 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:18.887968 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198c3f9c-a758-440f-89cf-6d8d26741238 admin admin] PATH_INFO: `/v3/projects/77495a957104457c8f0cbee8c01a4536` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:18.889102 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198c3f9c-a758-440f-89cf-6d8d26741238 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77495a957104457c8f0cbee8c01a4536)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:18.891067 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198c3f9c-a758-440f-89cf-6d8d26741238 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:19.033032 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-198c3f9c-a758-440f-89cf-6d8d26741238 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77495a957104457c8f0cbee8c01a4536', 'request_id': 'req-198c3f9c-a758-440f-89cf-6d8d26741238'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:19.038597 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-198c3f9c-a758-440f-89cf-6d8d26741238 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77495a957104457c8f0cbee8c01a4536', 'request_id': 'req-198c3f9c-a758-440f-89cf-6d8d26741238'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:19.045499 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-198c3f9c-a758-440f-89cf-6d8d26741238 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77495a957104457c8f0cbee8c01a4536', 'request_id': 'req-198c3f9c-a758-440f-89cf-6d8d26741238'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:19.130540 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-198c3f9c-a758-440f-89cf-6d8d26741238 admin admin] The token cache is being invalidate because project 77495a957104457c8f0cbee8c01a4536 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:00:19.130841 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-198c3f9c-a758-440f-89cf-6d8d26741238 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:19.131769 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-198c3f9c-a758-440f-89cf-6d8d26741238 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:19.133421 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1412/2793] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 23:00:18 2022] DELETE /identity/v3/projects/77495a957104457c8f0cbee8c01a4536 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:00:19.139009 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f395d-eb4c-4718-b81d-e7fdbf1384ed None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:19.139326 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f395d-eb4c-4718-b81d-e7fdbf1384ed None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:19.139603 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f395d-eb4c-4718-b81d-e7fdbf1384ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:19.160051 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-0a5f395d-eb4c-4718-b81d-e7fdbf1384ed None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:00:19.161401 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-0a5f395d-eb4c-4718-b81d-e7fdbf1384ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:19.181932 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:00:19.229205 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1382/2794] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 23:00:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:00:19.237972 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eacc0d-a54f-41ac-925c-6c2d8931a41a None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:19.238245 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eacc0d-a54f-41ac-925c-6c2d8931a41a None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:19.238522 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eacc0d-a54f-41ac-925c-6c2d8931a41a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:19.258991 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-52eacc0d-a54f-41ac-925c-6c2d8931a41a None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:00:19.260206 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-52eacc0d-a54f-41ac-925c-6c2d8931a41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:19.268955 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1413/2795] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 23:00:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:00:19.273301 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f395d-eb4c-4718-b81d-e7fdbf1384ed None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:19.283134 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f395d-eb4c-4718-b81d-e7fdbf1384ed None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:19.287882 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8cab70-88f5-475b-aad4-2a576a355923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:19.288483 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8cab70-88f5-475b-aad4-2a576a355923 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:19.288805 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8cab70-88f5-475b-aad4-2a576a355923 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:19.289092 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8cab70-88f5-475b-aad4-2a576a355923 admin admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:19.289729 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8cab70-88f5-475b-aad4-2a576a355923 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:00:19.290059 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8cab70-88f5-475b-aad4-2a576a355923 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:19.291702 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8cab70-88f5-475b-aad4-2a576a355923 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:19.303785 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1383/2796] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 23:00:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:00:19.308843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eacc0d-a54f-41ac-925c-6c2d8931a41a None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:19.319285 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eacc0d-a54f-41ac-925c-6c2d8931a41a None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:19.325134 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b80b9c9-6c84-448b-a492-44e5b6878fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:19.325737 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b80b9c9-6c84-448b-a492-44e5b6878fe7 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:19.326013 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b80b9c9-6c84-448b-a492-44e5b6878fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:19.326282 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b80b9c9-6c84-448b-a492-44e5b6878fe7 admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:19.326920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b80b9c9-6c84-448b-a492-44e5b6878fe7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:19.329062 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b80b9c9-6c84-448b-a492-44e5b6878fe7 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:19.341303 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:00:19.442236 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1414/2797] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 23:00:19 2022] POST /identity/v3/projects => generated 405 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:00:19.446454 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8cab70-88f5-475b-aad4-2a576a355923 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:19.456998 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8cab70-88f5-475b-aad4-2a576a355923 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:19.514882 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd6896-3235-4c81-87f1-019412387867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:19.515478 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd6896-3235-4c81-87f1-019412387867 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:19.515755 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd6896-3235-4c81-87f1-019412387867 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:19.516028 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd6896-3235-4c81-87f1-019412387867 admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:19.516737 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabd6896-3235-4c81-87f1-019412387867 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:19.518751 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabd6896-3235-4c81-87f1-019412387867 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:19.620857 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1384/2798] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 23:00:19 2022] POST /identity/v3/users => generated 303 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:00:19.623620 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b80b9c9-6c84-448b-a492-44e5b6878fe7 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:19.633952 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b80b9c9-6c84-448b-a492-44e5b6878fe7 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:19.638679 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041ecd4f-51a8-4d48-a0ea-aff7dda1b0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:19.639289 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ecd4f-51a8-4d48-a0ea-aff7dda1b0f5 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:19.639578 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ecd4f-51a8-4d48-a0ea-aff7dda1b0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:19.639853 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ecd4f-51a8-4d48-a0ea-aff7dda1b0f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:19.640493 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041ecd4f-51a8-4d48-a0ea-aff7dda1b0f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:19.642149 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041ecd4f-51a8-4d48-a0ea-aff7dda1b0f5 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:19.653031 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1415/2799] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 23:00:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:00:19.657121 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd6896-3235-4c81-87f1-019412387867 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:19.668639 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd6896-3235-4c81-87f1-019412387867 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:19.673456 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cbd027-7a9e-4911-becc-a9318727e99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:19.674057 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cbd027-7a9e-4911-becc-a9318727e99a admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:19.678322 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cbd027-7a9e-4911-becc-a9318727e99a admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:19.678322 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cbd027-7a9e-4911-becc-a9318727e99a admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:19.678322 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cbd027-7a9e-4911-becc-a9318727e99a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:19.678322 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cbd027-7a9e-4911-becc-a9318727e99a admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:19.690086 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1385/2800] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 23:00:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:00:19.694249 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041ecd4f-51a8-4d48-a0ea-aff7dda1b0f5 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:19.704347 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041ecd4f-51a8-4d48-a0ea-aff7dda1b0f5 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:19.709258 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b321460b-99c0-4744-8eec-9109e9be14be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:19.709893 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321460b-99c0-4744-8eec-9109e9be14be admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:19.710170 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321460b-99c0-4744-8eec-9109e9be14be admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:19.710440 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321460b-99c0-4744-8eec-9109e9be14be admin admin] PATH_INFO: `/v3/projects/9a975d21a380406caaccc45611aa4bef/users/876894a7e93b4c588d87b178d8a35f38/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:19.726336 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b321460b-99c0-4744-8eec-9109e9be14be admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a975d21a380406caaccc45611aa4bef, user_id=876894a7e93b4c588d87b178d8a35f38, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:19.729040 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b321460b-99c0-4744-8eec-9109e9be14be admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:19.810700 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1416/2801] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 23:00:19 2022] PUT /identity/v3/projects/9a975d21a380406caaccc45611aa4bef/users/876894a7e93b4c588d87b178d8a35f38/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:00:19.821218 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f42c6-8461-4eb5-a4fe-fb4eab4bfa17 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:19.821736 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f42c6-8461-4eb5-a4fe-fb4eab4bfa17 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:19.822085 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f42c6-8461-4eb5-a4fe-fb4eab4bfa17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:19.852070 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-8b2f42c6-8461-4eb5-a4fe-fb4eab4bfa17 None None] MFA Rules not processed for user `876894a7e93b4c588d87b178d8a35f38`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:00:19.854149 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-8b2f42c6-8461-4eb5-a4fe-fb4eab4bfa17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:19.926020 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1386/2802] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 23:00:19 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:00:19.930674 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b321460b-99c0-4744-8eec-9109e9be14be admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:19.993151 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b321460b-99c0-4744-8eec-9109e9be14be admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:19.997927 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69efa8c-5248-454e-ad65-ed28e79bcbd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:19.998532 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69efa8c-5248-454e-ad65-ed28e79bcbd3 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:19.998810 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69efa8c-5248-454e-ad65-ed28e79bcbd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:19.999085 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69efa8c-5248-454e-ad65-ed28e79bcbd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:19.999711 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69efa8c-5248-454e-ad65-ed28e79bcbd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:20.001680 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69efa8c-5248-454e-ad65-ed28e79bcbd3 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:20.123243 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1417/2803] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 23:00:19 2022] POST /identity/v3/projects => generated 405 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:00:20.127459 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f42c6-8461-4eb5-a4fe-fb4eab4bfa17 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:20.139939 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f42c6-8461-4eb5-a4fe-fb4eab4bfa17 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:20.164612 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:00:20.220482 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef01c6-8486-4633-9691-b5ba2a5a2002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:20.221349 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef01c6-8486-4633-9691-b5ba2a5a2002 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:20.221630 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef01c6-8486-4633-9691-b5ba2a5a2002 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:20.221907 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef01c6-8486-4633-9691-b5ba2a5a2002 admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:20.222581 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bef01c6-8486-4633-9691-b5ba2a5a2002 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:20.224669 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bef01c6-8486-4633-9691-b5ba2a5a2002 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:20.327316 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1387/2804] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 23:00:20 2022] POST /identity/v3/users => generated 309 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:00:20.331693 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69efa8c-5248-454e-ad65-ed28e79bcbd3 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:20.343011 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69efa8c-5248-454e-ad65-ed28e79bcbd3 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:20.347724 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e3b63-9a68-4bfb-ba57-7fd455a60eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:20.348323 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e3b63-9a68-4bfb-ba57-7fd455a60eaa admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:20.348605 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e3b63-9a68-4bfb-ba57-7fd455a60eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:20.348911 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e3b63-9a68-4bfb-ba57-7fd455a60eaa admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:20.349553 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128e3b63-9a68-4bfb-ba57-7fd455a60eaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:20.351150 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128e3b63-9a68-4bfb-ba57-7fd455a60eaa admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:20.361824 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1418/2805] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 23:00:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:00:20.366030 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef01c6-8486-4633-9691-b5ba2a5a2002 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:20.376420 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef01c6-8486-4633-9691-b5ba2a5a2002 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:20.381141 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674fc04b-063e-4724-8512-e121d70a469c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:20.381786 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674fc04b-063e-4724-8512-e121d70a469c admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:20.382070 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674fc04b-063e-4724-8512-e121d70a469c admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:20.382347 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674fc04b-063e-4724-8512-e121d70a469c admin admin] PATH_INFO: `/v3/projects/00fbb5c3c58d4b348a2c76814abb09f9/users/4cbcdc7159054a0b99aa5a0febf77387/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:20.398517 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674fc04b-063e-4724-8512-e121d70a469c admin admin] RBAC: Authorizing `identity:create_grant(project_id=00fbb5c3c58d4b348a2c76814abb09f9, user_id=4cbcdc7159054a0b99aa5a0febf77387, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:20.402131 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674fc04b-063e-4724-8512-e121d70a469c admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:20.499989 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1388/2806] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 23:00:20 2022] PUT /identity/v3/projects/00fbb5c3c58d4b348a2c76814abb09f9/users/4cbcdc7159054a0b99aa5a0febf77387/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:00:20.505371 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e3b63-9a68-4bfb-ba57-7fd455a60eaa admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:20.517323 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e3b63-9a68-4bfb-ba57-7fd455a60eaa admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:20.579805 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5885ec97-eb4a-402e-b708-9e92b600c5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:20.580406 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5885ec97-eb4a-402e-b708-9e92b600c5a7 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:20.580685 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5885ec97-eb4a-402e-b708-9e92b600c5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:20.580987 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5885ec97-eb4a-402e-b708-9e92b600c5a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:20.581629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5885ec97-eb4a-402e-b708-9e92b600c5a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:20.583241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5885ec97-eb4a-402e-b708-9e92b600c5a7 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:20.593862 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1419/2807] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 23:00:20 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:00:20.598216 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674fc04b-063e-4724-8512-e121d70a469c admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:20.608484 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674fc04b-063e-4724-8512-e121d70a469c admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:20.613148 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff90314-19cc-4ebb-9f12-d1d4052348c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:20.613792 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff90314-19cc-4ebb-9f12-d1d4052348c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:20.614069 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff90314-19cc-4ebb-9f12-d1d4052348c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:20.614342 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff90314-19cc-4ebb-9f12-d1d4052348c5 admin admin] PATH_INFO: `/v3/projects/00fbb5c3c58d4b348a2c76814abb09f9/users/4cbcdc7159054a0b99aa5a0febf77387/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:20.616228 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff90314-19cc-4ebb-9f12-d1d4052348c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00fbb5c3c58d4b348a2c76814abb09f9, user_id=4cbcdc7159054a0b99aa5a0febf77387, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:20.618879 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff90314-19cc-4ebb-9f12-d1d4052348c5 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:20.629719 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1389/2808] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 23:00:20 2022] PUT /identity/v3/projects/00fbb5c3c58d4b348a2c76814abb09f9/users/4cbcdc7159054a0b99aa5a0febf77387/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:00:20.641488 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24967051-4d53-446b-ad9d-ce4e3b255163 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:20.641773 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24967051-4d53-446b-ad9d-ce4e3b255163 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:20.642049 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24967051-4d53-446b-ad9d-ce4e3b255163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:20.672131 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-24967051-4d53-446b-ad9d-ce4e3b255163 None None] MFA Rules not processed for user `4cbcdc7159054a0b99aa5a0febf77387`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:00:20.673302 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-24967051-4d53-446b-ad9d-ce4e3b255163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:20.678643 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:00:20.745960 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1420/2809] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 23:00:20 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:00:20.759123 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff90314-19cc-4ebb-9f12-d1d4052348c5 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:20.762795 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-5ff90314-19cc-4ebb-9f12-d1d4052348c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:20.826570 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff90314-19cc-4ebb-9f12-d1d4052348c5 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:20.830218 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3845228e-9c0b-4ce0-8c93-0776a18fc626 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:20.830821 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3845228e-9c0b-4ce0-8c93-0776a18fc626 service cinder] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:20.831115 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3845228e-9c0b-4ce0-8c93-0776a18fc626 service cinder] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:20.831368 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3845228e-9c0b-4ce0-8c93-0776a18fc626 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:20.842422 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3845228e-9c0b-4ce0-8c93-0776a18fc626 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:20.844075 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3845228e-9c0b-4ce0-8c93-0776a18fc626 service cinder] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:20.856016 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1390/2810] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 23:00:20 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:00:21.008652 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24967051-4d53-446b-ad9d-ce4e3b255163 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:21.075202 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24967051-4d53-446b-ad9d-ce4e3b255163 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:21.079857 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d14ab9-7a4a-4993-ae7c-19a6945110ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:21.083942 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d14ab9-7a4a-4993-ae7c-19a6945110ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:21.083942 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d14ab9-7a4a-4993-ae7c-19a6945110ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:21.083942 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d14ab9-7a4a-4993-ae7c-19a6945110ba admin admin] PATH_INFO: `/v3/users/876894a7e93b4c588d87b178d8a35f38` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:21.083942 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d14ab9-7a4a-4993-ae7c-19a6945110ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=876894a7e93b4c588d87b178d8a35f38)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:21.084478 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d14ab9-7a4a-4993-ae7c-19a6945110ba admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:21.287227 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-a0d14ab9-7a4a-4993-ae7c-19a6945110ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '876894a7e93b4c588d87b178d8a35f38'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:21.410797 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-a0d14ab9-7a4a-4993-ae7c-19a6945110ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '876894a7e93b4c588d87b178d8a35f38'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:21.426588 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-a0d14ab9-7a4a-4993-ae7c-19a6945110ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '876894a7e93b4c588d87b178d8a35f38'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:21.434632 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-a0d14ab9-7a4a-4993-ae7c-19a6945110ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '876894a7e93b4c588d87b178d8a35f38'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:21.435342 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-a0d14ab9-7a4a-4993-ae7c-19a6945110ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '876894a7e93b4c588d87b178d8a35f38'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:21.441601 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1421/2811] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 23:00:21 2022] DELETE /identity/v3/users/876894a7e93b4c588d87b178d8a35f38 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:00:21.446397 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f8a35-5fc9-4b33-b4ae-b7da9ee80ebb None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:21.446687 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f8a35-5fc9-4b33-b4ae-b7da9ee80ebb None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:21.446962 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f8a35-5fc9-4b33-b4ae-b7da9ee80ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:21.466272 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-884f8a35-5fc9-4b33-b4ae-b7da9ee80ebb None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:00:21.468809 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-884f8a35-5fc9-4b33-b4ae-b7da9ee80ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:21.535990 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1391/2812] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 23:00:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:00:21.549442 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d14ab9-7a4a-4993-ae7c-19a6945110ba admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:21.551044 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-a0d14ab9-7a4a-4993-ae7c-19a6945110ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:21.583245 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:00:21.622048 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d14ab9-7a4a-4993-ae7c-19a6945110ba admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:21.630677 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd59d3-d423-4f57-87c7-2fc9cfda4609 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:21.631316 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd59d3-d423-4f57-87c7-2fc9cfda4609 service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:21.631593 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd59d3-d423-4f57-87c7-2fc9cfda4609 service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:21.631865 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd59d3-d423-4f57-87c7-2fc9cfda4609 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:21.642157 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd59d3-d423-4f57-87c7-2fc9cfda4609 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:21.643914 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd59d3-d423-4f57-87c7-2fc9cfda4609 service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:21.654538 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1422/2813] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 23:00:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:00:22.263938 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f8a35-5fc9-4b33-b4ae-b7da9ee80ebb None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:22.266083 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-884f8a35-5fc9-4b33-b4ae-b7da9ee80ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:22.287888 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f8a35-5fc9-4b33-b4ae-b7da9ee80ebb None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:22.293921 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ed14e-96cd-43f8-bca4-b8e708c39acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:22.294716 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ed14e-96cd-43f8-bca4-b8e708c39acc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:22.295067 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ed14e-96cd-43f8-bca4-b8e708c39acc admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:22.295358 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ed14e-96cd-43f8-bca4-b8e708c39acc admin admin] PATH_INFO: `/v3/projects/9a975d21a380406caaccc45611aa4bef` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:22.296858 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874ed14e-96cd-43f8-bca4-b8e708c39acc admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a975d21a380406caaccc45611aa4bef)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:22.298867 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874ed14e-96cd-43f8-bca4-b8e708c39acc admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:22.313012 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:00:22.427870 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-874ed14e-96cd-43f8-bca4-b8e708c39acc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a975d21a380406caaccc45611aa4bef', 'request_id': 'req-874ed14e-96cd-43f8-bca4-b8e708c39acc'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:22.496419 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-874ed14e-96cd-43f8-bca4-b8e708c39acc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a975d21a380406caaccc45611aa4bef', 'request_id': 'req-874ed14e-96cd-43f8-bca4-b8e708c39acc'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:22.516879 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-874ed14e-96cd-43f8-bca4-b8e708c39acc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a975d21a380406caaccc45611aa4bef', 'request_id': 'req-874ed14e-96cd-43f8-bca4-b8e708c39acc'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:22.531197 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-874ed14e-96cd-43f8-bca4-b8e708c39acc admin admin] The token cache is being invalidate because project 9a975d21a380406caaccc45611aa4bef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:00:22.531587 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-874ed14e-96cd-43f8-bca4-b8e708c39acc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:22.535937 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-874ed14e-96cd-43f8-bca4-b8e708c39acc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:22.536847 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1392/2814] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 23:00:22 2022] DELETE /identity/v3/projects/9a975d21a380406caaccc45611aa4bef => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:00:22.539536 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd59d3-d423-4f57-87c7-2fc9cfda4609 service neutron] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:22.559181 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd59d3-d423-4f57-87c7-2fc9cfda4609 service neutron] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:22.627348 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4d85f2-970d-42a0-adfb-b68c13f0156a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:22.630349 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4d85f2-970d-42a0-adfb-b68c13f0156a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:22.630349 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4d85f2-970d-42a0-adfb-b68c13f0156a admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:22.630349 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4d85f2-970d-42a0-adfb-b68c13f0156a admin admin] PATH_INFO: `/v3/users/4cbcdc7159054a0b99aa5a0febf77387` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:22.630349 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4d85f2-970d-42a0-adfb-b68c13f0156a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cbcdc7159054a0b99aa5a0febf77387)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:22.631750 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4d85f2-970d-42a0-adfb-b68c13f0156a admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:22.853777 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-9f4d85f2-970d-42a0-adfb-b68c13f0156a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cbcdc7159054a0b99aa5a0febf77387'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:22.924738 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-9f4d85f2-970d-42a0-adfb-b68c13f0156a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cbcdc7159054a0b99aa5a0febf77387'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:22.938532 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-9f4d85f2-970d-42a0-adfb-b68c13f0156a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cbcdc7159054a0b99aa5a0febf77387'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:22.945710 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-9f4d85f2-970d-42a0-adfb-b68c13f0156a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cbcdc7159054a0b99aa5a0febf77387'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:22.946427 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-9f4d85f2-970d-42a0-adfb-b68c13f0156a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cbcdc7159054a0b99aa5a0febf77387'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:22.953010 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1423/2815] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 23:00:22 2022] DELETE /identity/v3/users/4cbcdc7159054a0b99aa5a0febf77387 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:00:23.493286 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ed14e-96cd-43f8-bca4-b8e708c39acc admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:23.495383 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-874ed14e-96cd-43f8-bca4-b8e708c39acc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:23.508744 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ed14e-96cd-43f8-bca4-b8e708c39acc admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:23.569152 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d47ee-9bf0-4109-818e-5cc35d5eea25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:23.569784 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d47ee-9bf0-4109-818e-5cc35d5eea25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:23.570072 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d47ee-9bf0-4109-818e-5cc35d5eea25 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:23.570373 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d47ee-9bf0-4109-818e-5cc35d5eea25 admin admin] PATH_INFO: `/v3/projects/00fbb5c3c58d4b348a2c76814abb09f9` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:23.571517 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92d47ee-9bf0-4109-818e-5cc35d5eea25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00fbb5c3c58d4b348a2c76814abb09f9)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:23.573491 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92d47ee-9bf0-4109-818e-5cc35d5eea25 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:23.699029 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-f92d47ee-9bf0-4109-818e-5cc35d5eea25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00fbb5c3c58d4b348a2c76814abb09f9', 'request_id': 'req-f92d47ee-9bf0-4109-818e-5cc35d5eea25'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:23.781324 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-f92d47ee-9bf0-4109-818e-5cc35d5eea25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00fbb5c3c58d4b348a2c76814abb09f9', 'request_id': 'req-f92d47ee-9bf0-4109-818e-5cc35d5eea25'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:23.787350 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-f92d47ee-9bf0-4109-818e-5cc35d5eea25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00fbb5c3c58d4b348a2c76814abb09f9', 'request_id': 'req-f92d47ee-9bf0-4109-818e-5cc35d5eea25'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:23.799279 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-f92d47ee-9bf0-4109-818e-5cc35d5eea25 admin admin] The token cache is being invalidate because project 00fbb5c3c58d4b348a2c76814abb09f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:00:23.805822 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-f92d47ee-9bf0-4109-818e-5cc35d5eea25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:23.805822 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-f92d47ee-9bf0-4109-818e-5cc35d5eea25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:23.805822 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1393/2816] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 23:00:23 2022] DELETE /identity/v3/projects/00fbb5c3c58d4b348a2c76814abb09f9 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:00:23.806729 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e52ecf5-5cc2-4a16-9e6a-53077731b1c7 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:23.807219 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e52ecf5-5cc2-4a16-9e6a-53077731b1c7 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:23.807810 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e52ecf5-5cc2-4a16-9e6a-53077731b1c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:23.827540 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-1e52ecf5-5cc2-4a16-9e6a-53077731b1c7 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:00:23.828772 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-1e52ecf5-5cc2-4a16-9e6a-53077731b1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:23.871715 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:00:23.890485 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1424/2817] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 23:00:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:00:23.899496 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773a638-84c5-438d-adba-f897b1605870 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:23.899772 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773a638-84c5-438d-adba-f897b1605870 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:23.900047 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773a638-84c5-438d-adba-f897b1605870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:23.921410 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-b773a638-84c5-438d-adba-f897b1605870 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:00:23.922776 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-b773a638-84c5-438d-adba-f897b1605870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:23.930599 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1394/2818] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 23:00:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:00:23.935134 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e52ecf5-5cc2-4a16-9e6a-53077731b1c7 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:23.945430 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e52ecf5-5cc2-4a16-9e6a-53077731b1c7 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:23.951123 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34e27c-2e81-411c-b6a3-5233e84785d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:23.951123 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34e27c-2e81-411c-b6a3-5233e84785d2 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:23.951420 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34e27c-2e81-411c-b6a3-5233e84785d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:23.951552 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34e27c-2e81-411c-b6a3-5233e84785d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:23.952197 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c34e27c-2e81-411c-b6a3-5233e84785d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:00:23.952498 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c34e27c-2e81-411c-b6a3-5233e84785d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:23.954108 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c34e27c-2e81-411c-b6a3-5233e84785d2 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:23.966748 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1425/2819] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 23:00:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:00:23.971189 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b773a638-84c5-438d-adba-f897b1605870 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:23.981231 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b773a638-84c5-438d-adba-f897b1605870 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:23.985871 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40930a4e-ab8e-4d23-beb6-5274894306c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:23.986460 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40930a4e-ab8e-4d23-beb6-5274894306c4 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:23.986736 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40930a4e-ab8e-4d23-beb6-5274894306c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:23.987003 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40930a4e-ab8e-4d23-beb6-5274894306c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:23.987655 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40930a4e-ab8e-4d23-beb6-5274894306c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:23.989581 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40930a4e-ab8e-4d23-beb6-5274894306c4 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:24.059159 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:00:24.065485 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1395/2820] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 23:00:23 2022] POST /identity/v3/projects => generated 363 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:00:24.070190 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34e27c-2e81-411c-b6a3-5233e84785d2 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:24.080397 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34e27c-2e81-411c-b6a3-5233e84785d2 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:24.138000 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53157f6d-528f-4194-aa08-32bee94bd9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:24.138601 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53157f6d-528f-4194-aa08-32bee94bd9f2 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:24.138881 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53157f6d-528f-4194-aa08-32bee94bd9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:24.139155 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53157f6d-528f-4194-aa08-32bee94bd9f2 admin admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:24.139779 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53157f6d-528f-4194-aa08-32bee94bd9f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:24.141726 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53157f6d-528f-4194-aa08-32bee94bd9f2 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:24.265088 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1426/2821] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 23:00:24 2022] POST /identity/v3/users => generated 282 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:00:24.269511 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40930a4e-ab8e-4d23-beb6-5274894306c4 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:24.280907 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40930a4e-ab8e-4d23-beb6-5274894306c4 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:24.285618 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b0743e-3ba9-4e2d-bb1b-10816308e35a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:24.286221 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b0743e-3ba9-4e2d-bb1b-10816308e35a admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:24.286504 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b0743e-3ba9-4e2d-bb1b-10816308e35a admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:24.286784 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b0743e-3ba9-4e2d-bb1b-10816308e35a admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:24.287461 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b0743e-3ba9-4e2d-bb1b-10816308e35a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:24.289185 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b0743e-3ba9-4e2d-bb1b-10816308e35a admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:24.299835 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1396/2822] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 23:00:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:00:24.303989 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53157f6d-528f-4194-aa08-32bee94bd9f2 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:24.314241 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53157f6d-528f-4194-aa08-32bee94bd9f2 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:24.324210 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9413ebe9-6443-4e40-8a02-1b050c566fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:24.324847 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9413ebe9-6443-4e40-8a02-1b050c566fcd admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:24.325201 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9413ebe9-6443-4e40-8a02-1b050c566fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:24.330604 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9413ebe9-6443-4e40-8a02-1b050c566fcd admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:24.330604 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9413ebe9-6443-4e40-8a02-1b050c566fcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:24.330604 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9413ebe9-6443-4e40-8a02-1b050c566fcd admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:24.341057 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1427/2823] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 23:00:24 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:00:24.345358 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b0743e-3ba9-4e2d-bb1b-10816308e35a admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:24.355445 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b0743e-3ba9-4e2d-bb1b-10816308e35a admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:24.360024 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0031c79e-8d55-4329-8672-71d72871c7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:24.360666 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031c79e-8d55-4329-8672-71d72871c7f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:24.360986 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031c79e-8d55-4329-8672-71d72871c7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:24.361262 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031c79e-8d55-4329-8672-71d72871c7f5 admin admin] PATH_INFO: `/v3/projects/9e768d3863a34ecd90f7d72c031bdcf1/users/86243f83fb054508a6bca6f53c511ea2/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:24.375822 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0031c79e-8d55-4329-8672-71d72871c7f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e768d3863a34ecd90f7d72c031bdcf1, user_id=86243f83fb054508a6bca6f53c511ea2, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:24.378570 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0031c79e-8d55-4329-8672-71d72871c7f5 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:24.450976 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1397/2824] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 23:00:24 2022] PUT /identity/v3/projects/9e768d3863a34ecd90f7d72c031bdcf1/users/86243f83fb054508a6bca6f53c511ea2/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:00:24.461272 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01e19f-32ff-449f-b0f2-113e82cd5395 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:24.461558 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01e19f-32ff-449f-b0f2-113e82cd5395 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:24.461831 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01e19f-32ff-449f-b0f2-113e82cd5395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:24.491029 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-1b01e19f-32ff-449f-b0f2-113e82cd5395 None None] MFA Rules not processed for user `86243f83fb054508a6bca6f53c511ea2`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:00:24.492248 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-1b01e19f-32ff-449f-b0f2-113e82cd5395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:24.565442 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1428/2825] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 23:00:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:00:24.566622 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0031c79e-8d55-4329-8672-71d72871c7f5 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:24.645638 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0031c79e-8d55-4329-8672-71d72871c7f5 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:24.654015 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228a64ad-d3fc-4516-82df-ecd6a31ec857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:24.655055 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a64ad-d3fc-4516-82df-ecd6a31ec857 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:24.655561 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a64ad-d3fc-4516-82df-ecd6a31ec857 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:24.656067 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a64ad-d3fc-4516-82df-ecd6a31ec857 admin admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:24.657459 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228a64ad-d3fc-4516-82df-ecd6a31ec857 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:24.659349 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228a64ad-d3fc-4516-82df-ecd6a31ec857 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:24.761183 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1398/2826] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 23:00:24 2022] POST /identity/v3/projects => generated 365 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:00:24.765593 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b01e19f-32ff-449f-b0f2-113e82cd5395 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:24.776982 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b01e19f-32ff-449f-b0f2-113e82cd5395 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:24.815917 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:00:24.834463 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef0bb9-7ae1-45c8-bdcf-c9fdfabbb2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:24.835065 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef0bb9-7ae1-45c8-bdcf-c9fdfabbb2cb admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:24.835344 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef0bb9-7ae1-45c8-bdcf-c9fdfabbb2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:24.835619 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef0bb9-7ae1-45c8-bdcf-c9fdfabbb2cb admin admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:24.836255 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baef0bb9-7ae1-45c8-bdcf-c9fdfabbb2cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:24.838235 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baef0bb9-7ae1-45c8-bdcf-c9fdfabbb2cb admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:24.949391 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1429/2827] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 23:00:24 2022] POST /identity/v3/users => generated 289 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:00:24.953590 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228a64ad-d3fc-4516-82df-ecd6a31ec857 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:24.965514 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228a64ad-d3fc-4516-82df-ecd6a31ec857 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:24.970439 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71231b55-7094-4b0a-9cc5-85d529339e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:24.971043 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71231b55-7094-4b0a-9cc5-85d529339e83 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:24.971322 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71231b55-7094-4b0a-9cc5-85d529339e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:24.971595 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71231b55-7094-4b0a-9cc5-85d529339e83 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:24.972235 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71231b55-7094-4b0a-9cc5-85d529339e83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:24.973884 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71231b55-7094-4b0a-9cc5-85d529339e83 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:24.984926 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1399/2828] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 23:00:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:00:24.989434 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef0bb9-7ae1-45c8-bdcf-c9fdfabbb2cb admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:24.999848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef0bb9-7ae1-45c8-bdcf-c9fdfabbb2cb admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:25.004516 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e3e79-9cc1-45e8-a9b7-b751bf7c359d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:25.005196 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e3e79-9cc1-45e8-a9b7-b751bf7c359d admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:25.005481 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e3e79-9cc1-45e8-a9b7-b751bf7c359d admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:25.005757 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e3e79-9cc1-45e8-a9b7-b751bf7c359d admin admin] PATH_INFO: `/v3/projects/03f83d62a34748a3acab53f0ab4cc8fe/users/6243a3b08dfb4d4bb66c71e33eaf298b/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:25.021269 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1e3e79-9cc1-45e8-a9b7-b751bf7c359d admin admin] RBAC: Authorizing `identity:create_grant(project_id=03f83d62a34748a3acab53f0ab4cc8fe, user_id=6243a3b08dfb4d4bb66c71e33eaf298b, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:25.023924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1e3e79-9cc1-45e8-a9b7-b751bf7c359d admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:25.097792 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1430/2829] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 23:00:24 2022] PUT /identity/v3/projects/03f83d62a34748a3acab53f0ab4cc8fe/users/6243a3b08dfb4d4bb66c71e33eaf298b/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:00:25.101741 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71231b55-7094-4b0a-9cc5-85d529339e83 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:25.112277 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71231b55-7094-4b0a-9cc5-85d529339e83 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:25.168169 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8497d-817b-4312-85a0-c0df21dab74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:25.168798 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8497d-817b-4312-85a0-c0df21dab74e admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:25.169081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8497d-817b-4312-85a0-c0df21dab74e admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:25.169352 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8497d-817b-4312-85a0-c0df21dab74e admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:25.169984 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa8497d-817b-4312-85a0-c0df21dab74e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:25.171577 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa8497d-817b-4312-85a0-c0df21dab74e admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:25.181207 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1400/2830] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 23:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:00:25.185390 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e3e79-9cc1-45e8-a9b7-b751bf7c359d admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:25.195637 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e3e79-9cc1-45e8-a9b7-b751bf7c359d admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:25.200820 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809aa867-c74c-470b-bdde-9e7ac40fcea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:25.201475 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809aa867-c74c-470b-bdde-9e7ac40fcea3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:25.201745 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809aa867-c74c-470b-bdde-9e7ac40fcea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:25.202020 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809aa867-c74c-470b-bdde-9e7ac40fcea3 admin admin] PATH_INFO: `/v3/projects/03f83d62a34748a3acab53f0ab4cc8fe/users/6243a3b08dfb4d4bb66c71e33eaf298b/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:25.204055 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809aa867-c74c-470b-bdde-9e7ac40fcea3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03f83d62a34748a3acab53f0ab4cc8fe, user_id=6243a3b08dfb4d4bb66c71e33eaf298b, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:25.206720 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809aa867-c74c-470b-bdde-9e7ac40fcea3 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:25.217939 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1431/2831] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 23:00:25 2022] PUT /identity/v3/projects/03f83d62a34748a3acab53f0ab4cc8fe/users/6243a3b08dfb4d4bb66c71e33eaf298b/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:00:25.228861 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9deac4-3283-48bc-816d-3dd7b4996f57 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:25.229183 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9deac4-3283-48bc-816d-3dd7b4996f57 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:25.229458 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9deac4-3283-48bc-816d-3dd7b4996f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:25.258456 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-fa9deac4-3283-48bc-816d-3dd7b4996f57 None None] MFA Rules not processed for user `6243a3b08dfb4d4bb66c71e33eaf298b`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:00:25.259677 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-fa9deac4-3283-48bc-816d-3dd7b4996f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:25.284884 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:00:25.329873 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1401/2832] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 23:00:25 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:00:25.341957 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809aa867-c74c-470b-bdde-9e7ac40fcea3 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:25.343787 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-809aa867-c74c-470b-bdde-9e7ac40fcea3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:25.404456 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809aa867-c74c-470b-bdde-9e7ac40fcea3 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:25.408097 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef4e029-6ced-4465-8198-e2bbc6e8266d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:25.408704 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4e029-6ced-4465-8198-e2bbc6e8266d service cinder] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:25.409012 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4e029-6ced-4465-8198-e2bbc6e8266d service cinder] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:25.409283 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4e029-6ced-4465-8198-e2bbc6e8266d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:25.418269 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef4e029-6ced-4465-8198-e2bbc6e8266d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:25.419688 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef4e029-6ced-4465-8198-e2bbc6e8266d service cinder] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:25.428944 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1432/2833] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 23:00:25 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:00:25.626631 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9deac4-3283-48bc-816d-3dd7b4996f57 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:25.637015 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9deac4-3283-48bc-816d-3dd7b4996f57 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:25.640563 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826256ca-9cfa-445a-814d-667a74c663f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:25.641198 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826256ca-9cfa-445a-814d-667a74c663f7 service cinder] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:25.642142 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826256ca-9cfa-445a-814d-667a74c663f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:25.642142 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826256ca-9cfa-445a-814d-667a74c663f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:25.706487 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826256ca-9cfa-445a-814d-667a74c663f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:25.708000 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826256ca-9cfa-445a-814d-667a74c663f7 service cinder] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:25.717637 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1402/2834] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 23:00:25 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:00:29.280493 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef4e029-6ced-4465-8198-e2bbc6e8266d service cinder] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:29.349713 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef4e029-6ced-4465-8198-e2bbc6e8266d service cinder] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:29.357666 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e51c74-f26c-4981-a539-82564216f7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:29.358317 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e51c74-f26c-4981-a539-82564216f7c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:29.358596 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e51c74-f26c-4981-a539-82564216f7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:29.360877 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e51c74-f26c-4981-a539-82564216f7c4 admin admin] PATH_INFO: `/v3/users/86243f83fb054508a6bca6f53c511ea2` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:29.360877 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e51c74-f26c-4981-a539-82564216f7c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86243f83fb054508a6bca6f53c511ea2)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:29.362140 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e51c74-f26c-4981-a539-82564216f7c4 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:29.524019 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:00:29.532972 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-67e51c74-f26c-4981-a539-82564216f7c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86243f83fb054508a6bca6f53c511ea2'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:29.600842 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-67e51c74-f26c-4981-a539-82564216f7c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86243f83fb054508a6bca6f53c511ea2'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:29.614340 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-67e51c74-f26c-4981-a539-82564216f7c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86243f83fb054508a6bca6f53c511ea2'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:29.622542 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-67e51c74-f26c-4981-a539-82564216f7c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86243f83fb054508a6bca6f53c511ea2'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:29.623213 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-67e51c74-f26c-4981-a539-82564216f7c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86243f83fb054508a6bca6f53c511ea2'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:29.631347 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1433/2835] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 23:00:29 2022] DELETE /identity/v3/users/86243f83fb054508a6bca6f53c511ea2 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:00:29.636274 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ec1b0-9061-4695-8f6a-3b91bf172021 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:29.636804 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ec1b0-9061-4695-8f6a-3b91bf172021 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:29.636915 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ec1b0-9061-4695-8f6a-3b91bf172021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:29.656491 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-846ec1b0-9061-4695-8f6a-3b91bf172021 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:00:29.661255 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-846ec1b0-9061-4695-8f6a-3b91bf172021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:29.716409 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1403/2836] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 23:00:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:00:29.726162 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e51c74-f26c-4981-a539-82564216f7c4 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:29.727593 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-67e51c74-f26c-4981-a539-82564216f7c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:29.784755 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e51c74-f26c-4981-a539-82564216f7c4 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:29.788457 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c765d-8cdc-4ef8-8bee-6190ab22a17c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:29.789136 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c765d-8cdc-4ef8-8bee-6190ab22a17c service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:29.789417 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c765d-8cdc-4ef8-8bee-6190ab22a17c service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:29.789694 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c765d-8cdc-4ef8-8bee-6190ab22a17c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:29.799614 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c765d-8cdc-4ef8-8bee-6190ab22a17c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:29.801089 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c765d-8cdc-4ef8-8bee-6190ab22a17c service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:29.811427 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1434/2837] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 23:00:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:00:30.282294 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846ec1b0-9061-4695-8f6a-3b91bf172021 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:30.284292 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-846ec1b0-9061-4695-8f6a-3b91bf172021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:30.298570 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846ec1b0-9061-4695-8f6a-3b91bf172021 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:30.303554 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb078a-2daa-4384-9f43-3e8529958caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:30.304189 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb078a-2daa-4384-9f43-3e8529958caa admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:30.304473 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb078a-2daa-4384-9f43-3e8529958caa admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:30.304777 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb078a-2daa-4384-9f43-3e8529958caa admin admin] PATH_INFO: `/v3/projects/9e768d3863a34ecd90f7d72c031bdcf1` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:30.305999 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eb078a-2daa-4384-9f43-3e8529958caa admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e768d3863a34ecd90f7d72c031bdcf1)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:30.308790 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eb078a-2daa-4384-9f43-3e8529958caa admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:30.444540 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-18eb078a-2daa-4384-9f43-3e8529958caa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e768d3863a34ecd90f7d72c031bdcf1', 'request_id': 'req-18eb078a-2daa-4384-9f43-3e8529958caa'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:30.449763 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:00:30.521037 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-18eb078a-2daa-4384-9f43-3e8529958caa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e768d3863a34ecd90f7d72c031bdcf1', 'request_id': 'req-18eb078a-2daa-4384-9f43-3e8529958caa'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:30.529116 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-18eb078a-2daa-4384-9f43-3e8529958caa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e768d3863a34ecd90f7d72c031bdcf1', 'request_id': 'req-18eb078a-2daa-4384-9f43-3e8529958caa'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:30.557872 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-18eb078a-2daa-4384-9f43-3e8529958caa admin admin] The token cache is being invalidate because project 9e768d3863a34ecd90f7d72c031bdcf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:00:30.558148 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-18eb078a-2daa-4384-9f43-3e8529958caa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:30.559081 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-18eb078a-2daa-4384-9f43-3e8529958caa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:30.560859 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1404/2838] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 23:00:30 2022] DELETE /identity/v3/projects/9e768d3863a34ecd90f7d72c031bdcf1 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:00:30.564998 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c765d-8cdc-4ef8-8bee-6190ab22a17c service neutron] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:30.574862 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c765d-8cdc-4ef8-8bee-6190ab22a17c service neutron] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:30.636772 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55456cd1-7fd9-4f7e-8904-915931402627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:30.637424 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55456cd1-7fd9-4f7e-8904-915931402627 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:30.637692 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55456cd1-7fd9-4f7e-8904-915931402627 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:30.637968 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55456cd1-7fd9-4f7e-8904-915931402627 admin admin] PATH_INFO: `/v3/users/6243a3b08dfb4d4bb66c71e33eaf298b` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:30.639803 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55456cd1-7fd9-4f7e-8904-915931402627 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6243a3b08dfb4d4bb66c71e33eaf298b)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:30.642010 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55456cd1-7fd9-4f7e-8904-915931402627 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:30.775522 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-55456cd1-7fd9-4f7e-8904-915931402627 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6243a3b08dfb4d4bb66c71e33eaf298b'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:30.837760 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-55456cd1-7fd9-4f7e-8904-915931402627 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6243a3b08dfb4d4bb66c71e33eaf298b'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:30.842894 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:00:30.851199 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-55456cd1-7fd9-4f7e-8904-915931402627 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6243a3b08dfb4d4bb66c71e33eaf298b'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:30.858423 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-55456cd1-7fd9-4f7e-8904-915931402627 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6243a3b08dfb4d4bb66c71e33eaf298b'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:30.859103 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-55456cd1-7fd9-4f7e-8904-915931402627 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6243a3b08dfb4d4bb66c71e33eaf298b'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:30.865229 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1435/2839] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 23:00:30 2022] DELETE /identity/v3/users/6243a3b08dfb4d4bb66c71e33eaf298b => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:00:31.316796 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb078a-2daa-4384-9f43-3e8529958caa admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:31.318456 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-18eb078a-2daa-4384-9f43-3e8529958caa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:31.332244 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb078a-2daa-4384-9f43-3e8529958caa admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:31.393893 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de898e9-7003-4751-a9f7-0c191c77c4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:31.394530 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de898e9-7003-4751-a9f7-0c191c77c4bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:31.394812 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de898e9-7003-4751-a9f7-0c191c77c4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:31.395090 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de898e9-7003-4751-a9f7-0c191c77c4bd admin admin] PATH_INFO: `/v3/projects/03f83d62a34748a3acab53f0ab4cc8fe` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:31.396215 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de898e9-7003-4751-a9f7-0c191c77c4bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=03f83d62a34748a3acab53f0ab4cc8fe)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:31.398291 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de898e9-7003-4751-a9f7-0c191c77c4bd admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:31.495975 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-9de898e9-7003-4751-a9f7-0c191c77c4bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03f83d62a34748a3acab53f0ab4cc8fe', 'request_id': 'req-9de898e9-7003-4751-a9f7-0c191c77c4bd'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:31.558529 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-9de898e9-7003-4751-a9f7-0c191c77c4bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03f83d62a34748a3acab53f0ab4cc8fe', 'request_id': 'req-9de898e9-7003-4751-a9f7-0c191c77c4bd'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:31.564842 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-9de898e9-7003-4751-a9f7-0c191c77c4bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03f83d62a34748a3acab53f0ab4cc8fe', 'request_id': 'req-9de898e9-7003-4751-a9f7-0c191c77c4bd'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:31.576101 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-9de898e9-7003-4751-a9f7-0c191c77c4bd admin admin] The token cache is being invalidate because project 03f83d62a34748a3acab53f0ab4cc8fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:00:31.576393 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-9de898e9-7003-4751-a9f7-0c191c77c4bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:31.577270 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-9de898e9-7003-4751-a9f7-0c191c77c4bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:31.578876 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1405/2840] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 23:00:31 2022] DELETE /identity/v3/projects/03f83d62a34748a3acab53f0ab4cc8fe => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:00:31.584294 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7a91c-1830-4a70-99b8-67d29b23491c None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:31.584807 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7a91c-1830-4a70-99b8-67d29b23491c None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:31.584987 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7a91c-1830-4a70-99b8-67d29b23491c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:31.607670 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-07c7a91c-1830-4a70-99b8-67d29b23491c None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:00:31.608930 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-07c7a91c-1830-4a70-99b8-67d29b23491c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:31.676768 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1436/2841] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 23:00:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:00:31.678958 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc5645-0a6d-4a75-80d7-38efaf00a23e None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:31.679244 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc5645-0a6d-4a75-80d7-38efaf00a23e None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:31.679520 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc5645-0a6d-4a75-80d7-38efaf00a23e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:31.701201 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-d6dc5645-0a6d-4a75-80d7-38efaf00a23e None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:00:31.702421 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-d6dc5645-0a6d-4a75-80d7-38efaf00a23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:31.710815 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1406/2842] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 23:00:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:00:31.715491 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c7a91c-1830-4a70-99b8-67d29b23491c None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:31.725377 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c7a91c-1830-4a70-99b8-67d29b23491c None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:31.731234 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068fdcd1-ab9c-443b-ba66-16c1ada35440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:31.731861 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fdcd1-ab9c-443b-ba66-16c1ada35440 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:31.732138 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fdcd1-ab9c-443b-ba66-16c1ada35440 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:31.732406 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fdcd1-ab9c-443b-ba66-16c1ada35440 admin admin] PATH_INFO: `/v3/domains` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:31.733184 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068fdcd1-ab9c-443b-ba66-16c1ada35440 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:00:31.733484 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068fdcd1-ab9c-443b-ba66-16c1ada35440 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:31.735058 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068fdcd1-ab9c-443b-ba66-16c1ada35440 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:31.747600 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1437/2843] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 23:00:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:00:31.751983 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc5645-0a6d-4a75-80d7-38efaf00a23e None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:31.761357 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc5645-0a6d-4a75-80d7-38efaf00a23e None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:31.766069 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d2a1da-bdb3-4ae9-8090-105655b5f344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:31.766668 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d2a1da-bdb3-4ae9-8090-105655b5f344 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:31.766950 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d2a1da-bdb3-4ae9-8090-105655b5f344 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:31.767227 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d2a1da-bdb3-4ae9-8090-105655b5f344 admin admin] PATH_INFO: `/v3/projects` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:31.767854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d2a1da-bdb3-4ae9-8090-105655b5f344 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:31.769792 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d2a1da-bdb3-4ae9-8090-105655b5f344 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:31.852830 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1407/2844] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 23:00:31 2022] POST /identity/v3/projects => generated 391 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:00:31.858286 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068fdcd1-ab9c-443b-ba66-16c1ada35440 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:31.868043 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068fdcd1-ab9c-443b-ba66-16c1ada35440 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:31.926525 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6712947a-34a4-44ed-9d6f-e47edc57d812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:31.927129 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712947a-34a4-44ed-9d6f-e47edc57d812 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:31.927547 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712947a-34a4-44ed-9d6f-e47edc57d812 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:31.927664 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712947a-34a4-44ed-9d6f-e47edc57d812 admin admin] PATH_INFO: `/v3/users` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:31.928278 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6712947a-34a4-44ed-9d6f-e47edc57d812 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:31.930214 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6712947a-34a4-44ed-9d6f-e47edc57d812 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:32.033528 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1438/2845] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 23:00:31 2022] POST /identity/v3/users => generated 296 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:00:32.038016 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d2a1da-bdb3-4ae9-8090-105655b5f344 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:32.049278 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d2a1da-bdb3-4ae9-8090-105655b5f344 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:32.054139 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b1fd51-0121-4571-8baf-18973abcf7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:32.055086 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1fd51-0121-4571-8baf-18973abcf7fb admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:32.055395 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1fd51-0121-4571-8baf-18973abcf7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:32.055700 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1fd51-0121-4571-8baf-18973abcf7fb admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:32.056351 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b1fd51-0121-4571-8baf-18973abcf7fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:32.059031 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b1fd51-0121-4571-8baf-18973abcf7fb admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:32.069414 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1408/2846] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 23:00:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:00:32.073529 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6712947a-34a4-44ed-9d6f-e47edc57d812 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:32.084111 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6712947a-34a4-44ed-9d6f-e47edc57d812 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:32.089981 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b94e94-5818-4c0f-8646-1fcac89c53d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:32.090593 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b94e94-5818-4c0f-8646-1fcac89c53d8 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:32.090878 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b94e94-5818-4c0f-8646-1fcac89c53d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:32.091156 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b94e94-5818-4c0f-8646-1fcac89c53d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:32.091819 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b94e94-5818-4c0f-8646-1fcac89c53d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:32.093630 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b94e94-5818-4c0f-8646-1fcac89c53d8 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:32.105938 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1439/2847] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 23:00:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:00:32.110185 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b1fd51-0121-4571-8baf-18973abcf7fb admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:32.121111 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b94e94-5818-4c0f-8646-1fcac89c53d8 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:32.121728 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b1fd51-0121-4571-8baf-18973abcf7fb admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:32.122981 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-38b94e94-5818-4c0f-8646-1fcac89c53d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:32.126727 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12068862-fcef-4896-b007-120d57275e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:32.127387 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12068862-fcef-4896-b007-120d57275e2a admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:32.127662 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12068862-fcef-4896-b007-120d57275e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:32.127946 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12068862-fcef-4896-b007-120d57275e2a admin admin] PATH_INFO: `/v3/projects/c2c8d9f0457a4d6aaec9ea9fce8ca3c6/users/f4624e5bb58f49d8b4a3dd81439c8eae/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:32.144403 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12068862-fcef-4896-b007-120d57275e2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2c8d9f0457a4d6aaec9ea9fce8ca3c6, user_id=f4624e5bb58f49d8b4a3dd81439c8eae, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:32.147307 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12068862-fcef-4896-b007-120d57275e2a admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:32.177785 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:00:32.209137 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b94e94-5818-4c0f-8646-1fcac89c53d8 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:32.215477 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae7d526-cde8-478b-9cd6-aa7f817bd6b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:32.216089 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7d526-cde8-478b-9cd6-aa7f817bd6b7 service nova] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:32.216365 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7d526-cde8-478b-9cd6-aa7f817bd6b7 service nova] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:32.216640 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7d526-cde8-478b-9cd6-aa7f817bd6b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:32.218593 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-4ae7d526-cde8-478b-9cd6-aa7f817bd6b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:32.231853 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1409/2848] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 23:00:32 2022] PUT /identity/v3/projects/c2c8d9f0457a4d6aaec9ea9fce8ca3c6/users/f4624e5bb58f49d8b4a3dd81439c8eae/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:00:32.233055 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae7d526-cde8-478b-9cd6-aa7f817bd6b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:32.234795 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae7d526-cde8-478b-9cd6-aa7f817bd6b7 service nova] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:32.242600 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c680328a-2e4a-4360-985a-b5500fceb079 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:32.242933 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c680328a-2e4a-4360-985a-b5500fceb079 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:32.243269 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c680328a-2e4a-4360-985a-b5500fceb079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:32.247213 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1440/2849] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 23:00:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:00:32.266134 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:00:32.275148 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-c680328a-2e4a-4360-985a-b5500fceb079 None None] MFA Rules not processed for user `f4624e5bb58f49d8b4a3dd81439c8eae`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:00:32.276494 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-c680328a-2e4a-4360-985a-b5500fceb079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:32.350724 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1410/2850] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 23:00:32 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:00:32.355366 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae7d526-cde8-478b-9cd6-aa7f817bd6b7 service nova] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:32.365859 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae7d526-cde8-478b-9cd6-aa7f817bd6b7 service nova] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:32.426087 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1388b835-3ecf-4c51-b7b7-902ea8f4dfe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:32.426709 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1388b835-3ecf-4c51-b7b7-902ea8f4dfe3 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:32.426995 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1388b835-3ecf-4c51-b7b7-902ea8f4dfe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:32.427256 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1388b835-3ecf-4c51-b7b7-902ea8f4dfe3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:32.427903 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1388b835-3ecf-4c51-b7b7-902ea8f4dfe3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:32.429846 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1388b835-3ecf-4c51-b7b7-902ea8f4dfe3 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:32.529848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1441/2851] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 23:00:32 2022] POST /identity/v3/projects => generated 391 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:00:32.531805 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c680328a-2e4a-4360-985a-b5500fceb079 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:32.547679 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c680328a-2e4a-4360-985a-b5500fceb079 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:32.605794 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979ac2c9-de5b-4ecb-816c-b3f43127d0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:32.606407 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ac2c9-de5b-4ecb-816c-b3f43127d0d2 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:32.606764 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ac2c9-de5b-4ecb-816c-b3f43127d0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:32.606963 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ac2c9-de5b-4ecb-816c-b3f43127d0d2 admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:32.607609 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979ac2c9-de5b-4ecb-816c-b3f43127d0d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:32.609625 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979ac2c9-de5b-4ecb-816c-b3f43127d0d2 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:32.694361 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1411/2852] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 23:00:32 2022] POST /identity/v3/users => generated 302 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:00:32.698540 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1388b835-3ecf-4c51-b7b7-902ea8f4dfe3 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:32.710037 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1388b835-3ecf-4c51-b7b7-902ea8f4dfe3 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:32.715333 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f044c8-d551-4985-98cc-f60fa64d8889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:32.716070 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f044c8-d551-4985-98cc-f60fa64d8889 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:32.716758 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f044c8-d551-4985-98cc-f60fa64d8889 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:32.716758 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f044c8-d551-4985-98cc-f60fa64d8889 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:32.717361 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f044c8-d551-4985-98cc-f60fa64d8889 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:32.718991 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f044c8-d551-4985-98cc-f60fa64d8889 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:32.731216 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1442/2853] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 23:00:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:00:32.736393 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979ac2c9-de5b-4ecb-816c-b3f43127d0d2 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:32.744437 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f044c8-d551-4985-98cc-f60fa64d8889 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:32.748208 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-e2f044c8-d551-4985-98cc-f60fa64d8889 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:32.751085 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979ac2c9-de5b-4ecb-816c-b3f43127d0d2 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:32.766597 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fbc707-258c-4275-b00f-9d812bcea633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:32.767546 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbc707-258c-4275-b00f-9d812bcea633 admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:32.767827 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbc707-258c-4275-b00f-9d812bcea633 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:32.768100 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbc707-258c-4275-b00f-9d812bcea633 admin admin] PATH_INFO: `/v3/projects/c03001809cce49aaabf9070e7c6638b0/users/f4ee063e2f4042c58f3c95ea4cd192b1/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:32.788013 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fbc707-258c-4275-b00f-9d812bcea633 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c03001809cce49aaabf9070e7c6638b0, user_id=f4ee063e2f4042c58f3c95ea4cd192b1, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:32.789852 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fbc707-258c-4275-b00f-9d812bcea633 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:32.830192 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f044c8-d551-4985-98cc-f60fa64d8889 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:32.836261 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f371ef5-7bbe-4679-98fe-8e75f333130a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:32.836894 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f371ef5-7bbe-4679-98fe-8e75f333130a service nova] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:32.837173 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f371ef5-7bbe-4679-98fe-8e75f333130a service nova] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:32.837443 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f371ef5-7bbe-4679-98fe-8e75f333130a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:32.839791 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-2f371ef5-7bbe-4679-98fe-8e75f333130a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:32.878675 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:00:32.886682 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1412/2854] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 23:00:32 2022] PUT /identity/v3/projects/c03001809cce49aaabf9070e7c6638b0/users/f4ee063e2f4042c58f3c95ea4cd192b1/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:00:32.890733 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fbc707-258c-4275-b00f-9d812bcea633 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:32.899977 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f371ef5-7bbe-4679-98fe-8e75f333130a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:32.902205 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f371ef5-7bbe-4679-98fe-8e75f333130a service nova] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:32.911136 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1443/2855] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 23:00:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:00:32.958898 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fbc707-258c-4275-b00f-9d812bcea633 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:32.964260 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1bea89-2789-44a9-bd93-b79ecc21fa59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:32.964961 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1bea89-2789-44a9-bd93-b79ecc21fa59 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:32.965245 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1bea89-2789-44a9-bd93-b79ecc21fa59 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:32.965523 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1bea89-2789-44a9-bd93-b79ecc21fa59 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:32.966173 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1bea89-2789-44a9-bd93-b79ecc21fa59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:32.967822 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1bea89-2789-44a9-bd93-b79ecc21fa59 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:32.973557 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:00:32.979610 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1413/2856] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 23:00:32 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:00:32.984062 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f371ef5-7bbe-4679-98fe-8e75f333130a service nova] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:32.995885 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f371ef5-7bbe-4679-98fe-8e75f333130a service nova] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:33.001830 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a8d7ba-80f4-4495-9f0e-0dcb1a6c1d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:33.002646 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8d7ba-80f4-4495-9f0e-0dcb1a6c1d25 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:33.003511 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8d7ba-80f4-4495-9f0e-0dcb1a6c1d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:33.003511 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8d7ba-80f4-4495-9f0e-0dcb1a6c1d25 admin admin] PATH_INFO: `/v3/projects/c03001809cce49aaabf9070e7c6638b0/users/f4ee063e2f4042c58f3c95ea4cd192b1/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:33.009050 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a8d7ba-80f4-4495-9f0e-0dcb1a6c1d25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c03001809cce49aaabf9070e7c6638b0, user_id=f4ee063e2f4042c58f3c95ea4cd192b1, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:33.013260 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a8d7ba-80f4-4495-9f0e-0dcb1a6c1d25 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:33.024773 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1444/2857] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 23:00:32 2022] PUT /identity/v3/projects/c03001809cce49aaabf9070e7c6638b0/users/f4ee063e2f4042c58f3c95ea4cd192b1/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:00:33.100317 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79064c62-0cbf-4e63-a771-0a0d5b2a78b6 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:33.100604 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79064c62-0cbf-4e63-a771-0a0d5b2a78b6 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:33.100908 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79064c62-0cbf-4e63-a771-0a0d5b2a78b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:33.134986 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-79064c62-0cbf-4e63-a771-0a0d5b2a78b6 None None] MFA Rules not processed for user `f4ee063e2f4042c58f3c95ea4cd192b1`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:00:33.134986 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-79064c62-0cbf-4e63-a771-0a0d5b2a78b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:33.202447 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1414/2858] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 23:00:33 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:00:33.214659 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a8d7ba-80f4-4495-9f0e-0dcb1a6c1d25 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:33.216369 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-f3a8d7ba-80f4-4495-9f0e-0dcb1a6c1d25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:33.274692 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a8d7ba-80f4-4495-9f0e-0dcb1a6c1d25 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:33.278888 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db80b56e-3e5d-406f-9495-2e0a57b54d9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:33.279490 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db80b56e-3e5d-406f-9495-2e0a57b54d9b service cinder] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:33.279771 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db80b56e-3e5d-406f-9495-2e0a57b54d9b service cinder] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:33.280056 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db80b56e-3e5d-406f-9495-2e0a57b54d9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:33.289076 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db80b56e-3e5d-406f-9495-2e0a57b54d9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:33.290500 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db80b56e-3e5d-406f-9495-2e0a57b54d9b service cinder] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:33.300493 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1445/2859] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 23:00:33 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:00:33.348849 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79064c62-0cbf-4e63-a771-0a0d5b2a78b6 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:33.350444 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-79064c62-0cbf-4e63-a771-0a0d5b2a78b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:33.363599 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79064c62-0cbf-4e63-a771-0a0d5b2a78b6 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:33.367453 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f07abc-fcc8-469e-a32e-eaf6e17ea941 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:33.368049 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f07abc-fcc8-469e-a32e-eaf6e17ea941 service cinder] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:33.368331 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f07abc-fcc8-469e-a32e-eaf6e17ea941 service cinder] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:33.368601 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f07abc-fcc8-469e-a32e-eaf6e17ea941 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:33.427399 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f07abc-fcc8-469e-a32e-eaf6e17ea941 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:33.428774 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f07abc-fcc8-469e-a32e-eaf6e17ea941 service cinder] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:33.438279 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1415/2860] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 23:00:33 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:00:37.664884 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db80b56e-3e5d-406f-9495-2e0a57b54d9b service cinder] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:37.753175 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db80b56e-3e5d-406f-9495-2e0a57b54d9b service cinder] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:37.759307 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c717843-c5c3-4f2f-aa26-64b8b580f22a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbf961e02934e88af3a72d6218f32fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:37.760503 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c717843-c5c3-4f2f-aa26-64b8b580f22a service nova] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:37.760503 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c717843-c5c3-4f2f-aa26-64b8b580f22a service nova] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:37.761073 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c717843-c5c3-4f2f-aa26-64b8b580f22a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:37.762566 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-7c717843-c5c3-4f2f-aa26-64b8b580f22a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:37.775378 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c717843-c5c3-4f2f-aa26-64b8b580f22a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:37.776997 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c717843-c5c3-4f2f-aa26-64b8b580f22a service nova] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:37.788021 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1446/2861] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 23:00:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:00:38.230360 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f07abc-fcc8-469e-a32e-eaf6e17ea941 service cinder] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:38.311127 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f07abc-fcc8-469e-a32e-eaf6e17ea941 service cinder] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:38.315916 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53be80f0-2d0e-49ce-a5a8-a3bf9db731e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:38.320517 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be80f0-2d0e-49ce-a5a8-a3bf9db731e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:38.320517 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be80f0-2d0e-49ce-a5a8-a3bf9db731e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:38.320517 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be80f0-2d0e-49ce-a5a8-a3bf9db731e8 admin admin] PATH_INFO: `/v3/users/f4624e5bb58f49d8b4a3dd81439c8eae` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:38.320517 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53be80f0-2d0e-49ce-a5a8-a3bf9db731e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4624e5bb58f49d8b4a3dd81439c8eae)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:38.320517 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53be80f0-2d0e-49ce-a5a8-a3bf9db731e8 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:38.418412 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:00:38.509420 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-53be80f0-2d0e-49ce-a5a8-a3bf9db731e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4624e5bb58f49d8b4a3dd81439c8eae'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:38.522191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-53be80f0-2d0e-49ce-a5a8-a3bf9db731e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4624e5bb58f49d8b4a3dd81439c8eae'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:38.529642 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-53be80f0-2d0e-49ce-a5a8-a3bf9db731e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4624e5bb58f49d8b4a3dd81439c8eae'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:38.582970 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-53be80f0-2d0e-49ce-a5a8-a3bf9db731e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4624e5bb58f49d8b4a3dd81439c8eae'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:38.583868 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-53be80f0-2d0e-49ce-a5a8-a3bf9db731e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4624e5bb58f49d8b4a3dd81439c8eae'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:38.590718 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1416/2862] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 23:00:38 2022] DELETE /identity/v3/users/f4624e5bb58f49d8b4a3dd81439c8eae => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:00:38.595474 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c0896-2bf7-4e1f-a490-c98d570828a8 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:38.595762 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c0896-2bf7-4e1f-a490-c98d570828a8 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:38.600636 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c0896-2bf7-4e1f-a490-c98d570828a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:38.615923 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-ab8c0896-2bf7-4e1f-a490-c98d570828a8 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:00:38.617157 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-ab8c0896-2bf7-4e1f-a490-c98d570828a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:38.666564 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:00:38.679413 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1447/2863] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 23:00:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:00:38.690260 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53be80f0-2d0e-49ce-a5a8-a3bf9db731e8 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:38.691700 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-53be80f0-2d0e-49ce-a5a8-a3bf9db731e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:38.751921 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53be80f0-2d0e-49ce-a5a8-a3bf9db731e8 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:38.755921 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573a80fd-5697-45ef-a5c1-2828e133b6f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:38.756540 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a80fd-5697-45ef-a5c1-2828e133b6f4 service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:38.756863 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a80fd-5697-45ef-a5c1-2828e133b6f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:38.757138 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a80fd-5697-45ef-a5c1-2828e133b6f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:38.768435 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573a80fd-5697-45ef-a5c1-2828e133b6f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:38.769910 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573a80fd-5697-45ef-a5c1-2828e133b6f4 service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:38.781448 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1417/2864] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 23:00:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:00:39.184863 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c0896-2bf7-4e1f-a490-c98d570828a8 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:39.195233 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-ab8c0896-2bf7-4e1f-a490-c98d570828a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:39.199101 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c0896-2bf7-4e1f-a490-c98d570828a8 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:39.204242 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39623842-62c0-458a-9898-0a2cc73f79d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:39.204919 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39623842-62c0-458a-9898-0a2cc73f79d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:39.205202 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39623842-62c0-458a-9898-0a2cc73f79d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:39.205483 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39623842-62c0-458a-9898-0a2cc73f79d9 admin admin] PATH_INFO: `/v3/projects/c2c8d9f0457a4d6aaec9ea9fce8ca3c6` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:39.206621 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39623842-62c0-458a-9898-0a2cc73f79d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2c8d9f0457a4d6aaec9ea9fce8ca3c6)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:39.208934 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39623842-62c0-458a-9898-0a2cc73f79d9 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:39.318441 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-39623842-62c0-458a-9898-0a2cc73f79d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2c8d9f0457a4d6aaec9ea9fce8ca3c6', 'request_id': 'req-39623842-62c0-458a-9898-0a2cc73f79d9'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:39.325810 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-39623842-62c0-458a-9898-0a2cc73f79d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2c8d9f0457a4d6aaec9ea9fce8ca3c6', 'request_id': 'req-39623842-62c0-458a-9898-0a2cc73f79d9'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:39.332569 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-39623842-62c0-458a-9898-0a2cc73f79d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2c8d9f0457a4d6aaec9ea9fce8ca3c6', 'request_id': 'req-39623842-62c0-458a-9898-0a2cc73f79d9'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:39.403356 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-39623842-62c0-458a-9898-0a2cc73f79d9 admin admin] The token cache is being invalidate because project c2c8d9f0457a4d6aaec9ea9fce8ca3c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:00:39.403669 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-39623842-62c0-458a-9898-0a2cc73f79d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:39.404273 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-39623842-62c0-458a-9898-0a2cc73f79d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:39.405813 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1448/2865] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 23:00:39 2022] DELETE /identity/v3/projects/c2c8d9f0457a4d6aaec9ea9fce8ca3c6 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:00:39.409935 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573a80fd-5697-45ef-a5c1-2828e133b6f4 service neutron] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:39.420238 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573a80fd-5697-45ef-a5c1-2828e133b6f4 service neutron] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:39.477901 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc99030d-94d1-4b6e-9d3f-a0e8d1f0731b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:39.478960 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc99030d-94d1-4b6e-9d3f-a0e8d1f0731b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:39.479094 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc99030d-94d1-4b6e-9d3f-a0e8d1f0731b admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:39.479282 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc99030d-94d1-4b6e-9d3f-a0e8d1f0731b admin admin] PATH_INFO: `/v3/users/f4ee063e2f4042c58f3c95ea4cd192b1` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:39.480390 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc99030d-94d1-4b6e-9d3f-a0e8d1f0731b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4ee063e2f4042c58f3c95ea4cd192b1)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:39.482333 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc99030d-94d1-4b6e-9d3f-a0e8d1f0731b admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:39.646684 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-fc99030d-94d1-4b6e-9d3f-a0e8d1f0731b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4ee063e2f4042c58f3c95ea4cd192b1'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:39.647917 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:00:39.659309 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-fc99030d-94d1-4b6e-9d3f-a0e8d1f0731b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4ee063e2f4042c58f3c95ea4cd192b1'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:39.666199 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-fc99030d-94d1-4b6e-9d3f-a0e8d1f0731b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4ee063e2f4042c58f3c95ea4cd192b1'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:39.723551 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-fc99030d-94d1-4b6e-9d3f-a0e8d1f0731b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4ee063e2f4042c58f3c95ea4cd192b1'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:39.724311 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-fc99030d-94d1-4b6e-9d3f-a0e8d1f0731b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4ee063e2f4042c58f3c95ea4cd192b1'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:39.731125 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1418/2866] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 23:00:39 2022] DELETE /identity/v3/users/f4ee063e2f4042c58f3c95ea4cd192b1 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:00:40.209128 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39623842-62c0-458a-9898-0a2cc73f79d9 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:40.210595 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-39623842-62c0-458a-9898-0a2cc73f79d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:40.224895 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39623842-62c0-458a-9898-0a2cc73f79d9 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:40.286081 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eef9412-baa8-4ff2-bc69-cfc4052fcdb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:40.286629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef9412-baa8-4ff2-bc69-cfc4052fcdb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:40.286912 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef9412-baa8-4ff2-bc69-cfc4052fcdb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:40.287190 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef9412-baa8-4ff2-bc69-cfc4052fcdb6 admin admin] PATH_INFO: `/v3/projects/c03001809cce49aaabf9070e7c6638b0` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:40.288312 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eef9412-baa8-4ff2-bc69-cfc4052fcdb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c03001809cce49aaabf9070e7c6638b0)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:40.290263 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eef9412-baa8-4ff2-bc69-cfc4052fcdb6 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:40.444153 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-2eef9412-baa8-4ff2-bc69-cfc4052fcdb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c03001809cce49aaabf9070e7c6638b0', 'request_id': 'req-2eef9412-baa8-4ff2-bc69-cfc4052fcdb6'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:40.449612 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-2eef9412-baa8-4ff2-bc69-cfc4052fcdb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c03001809cce49aaabf9070e7c6638b0', 'request_id': 'req-2eef9412-baa8-4ff2-bc69-cfc4052fcdb6'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:40.456546 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-2eef9412-baa8-4ff2-bc69-cfc4052fcdb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c03001809cce49aaabf9070e7c6638b0', 'request_id': 'req-2eef9412-baa8-4ff2-bc69-cfc4052fcdb6'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:40.530179 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-2eef9412-baa8-4ff2-bc69-cfc4052fcdb6 admin admin] The token cache is being invalidate because project c03001809cce49aaabf9070e7c6638b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:00:40.530449 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-2eef9412-baa8-4ff2-bc69-cfc4052fcdb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:40.531138 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-2eef9412-baa8-4ff2-bc69-cfc4052fcdb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:40.532814 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1449/2867] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 23:00:40 2022] DELETE /identity/v3/projects/c03001809cce49aaabf9070e7c6638b0 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:00:40.540107 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135344d-258b-426b-9ba7-2ae2de75a608 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:40.541050 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135344d-258b-426b-9ba7-2ae2de75a608 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:40.541399 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135344d-258b-426b-9ba7-2ae2de75a608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:40.564070 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-3135344d-258b-426b-9ba7-2ae2de75a608 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:00:40.568913 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-3135344d-258b-426b-9ba7-2ae2de75a608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:40.636351 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1419/2868] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 23:00:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:00:40.644538 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb4a42d-2225-464d-b59b-19176091ca4d None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:40.644951 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb4a42d-2225-464d-b59b-19176091ca4d None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:40.645315 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb4a42d-2225-464d-b59b-19176091ca4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:40.661533 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:00:40.668184 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-6cb4a42d-2225-464d-b59b-19176091ca4d None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:00:40.674475 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-6cb4a42d-2225-464d-b59b-19176091ca4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:40.679223 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1450/2869] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 23:00:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:00:40.684333 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135344d-258b-426b-9ba7-2ae2de75a608 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:40.694695 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135344d-258b-426b-9ba7-2ae2de75a608 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:40.700934 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb7e95-837e-4754-bb71-b142ef705eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:40.701555 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb7e95-837e-4754-bb71-b142ef705eaf admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:40.701943 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb7e95-837e-4754-bb71-b142ef705eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:40.702168 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb7e95-837e-4754-bb71-b142ef705eaf admin admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:40.702833 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fb7e95-837e-4754-bb71-b142ef705eaf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:00:40.703138 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fb7e95-837e-4754-bb71-b142ef705eaf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:40.705191 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fb7e95-837e-4754-bb71-b142ef705eaf admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:40.718890 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1420/2870] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 23:00:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:00:40.723401 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb4a42d-2225-464d-b59b-19176091ca4d None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:40.733738 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb4a42d-2225-464d-b59b-19176091ca4d None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:40.739546 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18599b8-0484-42ef-b018-49c63702b0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:40.740357 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18599b8-0484-42ef-b018-49c63702b0e8 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:40.740843 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18599b8-0484-42ef-b018-49c63702b0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:40.741358 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18599b8-0484-42ef-b018-49c63702b0e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:40.742004 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18599b8-0484-42ef-b018-49c63702b0e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:40.743896 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18599b8-0484-42ef-b018-49c63702b0e8 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:40.835585 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1451/2871] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 23:00:40 2022] POST /identity/v3/projects => generated 377 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:00:40.840000 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb7e95-837e-4754-bb71-b142ef705eaf admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:40.850518 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb7e95-837e-4754-bb71-b142ef705eaf admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:40.907531 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331a196a-063b-4f77-a96e-47fdb251d35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:40.908137 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a196a-063b-4f77-a96e-47fdb251d35b admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:40.908420 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a196a-063b-4f77-a96e-47fdb251d35b admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:40.908698 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a196a-063b-4f77-a96e-47fdb251d35b admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:40.909374 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331a196a-063b-4f77-a96e-47fdb251d35b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:40.911526 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331a196a-063b-4f77-a96e-47fdb251d35b admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:40.992668 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1421/2872] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 23:00:40 2022] POST /identity/v3/users => generated 289 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:00:40.997667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18599b8-0484-42ef-b018-49c63702b0e8 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:41.008235 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18599b8-0484-42ef-b018-49c63702b0e8 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:41.012907 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781d6208-f2ea-4de5-839c-10e473aedfb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:41.013501 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d6208-f2ea-4de5-839c-10e473aedfb5 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:41.013783 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d6208-f2ea-4de5-839c-10e473aedfb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:41.019980 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d6208-f2ea-4de5-839c-10e473aedfb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:41.019980 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781d6208-f2ea-4de5-839c-10e473aedfb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:41.019980 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781d6208-f2ea-4de5-839c-10e473aedfb5 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:41.026281 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1452/2873] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 23:00:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:00:41.030304 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331a196a-063b-4f77-a96e-47fdb251d35b admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:41.040401 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331a196a-063b-4f77-a96e-47fdb251d35b admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:41.045657 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb6f64d-7313-4177-afa6-4ff4d3a7bd22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:41.046251 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb6f64d-7313-4177-afa6-4ff4d3a7bd22 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:41.046533 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb6f64d-7313-4177-afa6-4ff4d3a7bd22 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:41.046806 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb6f64d-7313-4177-afa6-4ff4d3a7bd22 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:41.047449 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb6f64d-7313-4177-afa6-4ff4d3a7bd22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:41.049074 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb6f64d-7313-4177-afa6-4ff4d3a7bd22 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:41.060919 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1422/2874] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 23:00:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:00:41.065042 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781d6208-f2ea-4de5-839c-10e473aedfb5 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:41.075335 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781d6208-f2ea-4de5-839c-10e473aedfb5 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:41.081146 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa449730-3a6f-4005-99f0-86fe24247600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:41.081798 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa449730-3a6f-4005-99f0-86fe24247600 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:41.082078 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa449730-3a6f-4005-99f0-86fe24247600 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:41.082354 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa449730-3a6f-4005-99f0-86fe24247600 admin admin] PATH_INFO: `/v3/projects/004855483053406bbe809eddf288a559/users/a53a9eae01064be99a9fda1f50b62358/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:41.098177 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa449730-3a6f-4005-99f0-86fe24247600 admin admin] RBAC: Authorizing `identity:create_grant(project_id=004855483053406bbe809eddf288a559, user_id=a53a9eae01064be99a9fda1f50b62358, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:41.100811 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa449730-3a6f-4005-99f0-86fe24247600 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:41.175305 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1453/2875] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 23:00:41 2022] PUT /identity/v3/projects/004855483053406bbe809eddf288a559/users/a53a9eae01064be99a9fda1f50b62358/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:00:41.185693 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f06426c-63ff-42b1-9488-d5e3ede59d34 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:41.186014 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f06426c-63ff-42b1-9488-d5e3ede59d34 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:41.186371 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f06426c-63ff-42b1-9488-d5e3ede59d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:41.215547 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-8f06426c-63ff-42b1-9488-d5e3ede59d34 None None] MFA Rules not processed for user `a53a9eae01064be99a9fda1f50b62358`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:00:41.216874 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-8f06426c-63ff-42b1-9488-d5e3ede59d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:41.237995 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:00:41.286074 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1423/2876] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 23:00:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:00:41.290633 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa449730-3a6f-4005-99f0-86fe24247600 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:41.352271 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa449730-3a6f-4005-99f0-86fe24247600 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:41.357548 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90de8034-523f-49de-8c78-9375304fcd49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:41.358144 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de8034-523f-49de-8c78-9375304fcd49 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:41.358432 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de8034-523f-49de-8c78-9375304fcd49 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:41.358707 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de8034-523f-49de-8c78-9375304fcd49 admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:41.359401 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90de8034-523f-49de-8c78-9375304fcd49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:41.361274 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90de8034-523f-49de-8c78-9375304fcd49 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:41.477882 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1454/2877] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 23:00:41 2022] POST /identity/v3/projects => generated 379 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:00:41.479843 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f06426c-63ff-42b1-9488-d5e3ede59d34 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:41.490870 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f06426c-63ff-42b1-9488-d5e3ede59d34 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:41.548057 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34fd9e3-2a59-412d-b7a3-ee5eb802ff51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:41.548754 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34fd9e3-2a59-412d-b7a3-ee5eb802ff51 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:41.548960 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34fd9e3-2a59-412d-b7a3-ee5eb802ff51 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:41.549231 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34fd9e3-2a59-412d-b7a3-ee5eb802ff51 admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:41.549869 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34fd9e3-2a59-412d-b7a3-ee5eb802ff51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:41.551783 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34fd9e3-2a59-412d-b7a3-ee5eb802ff51 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:41.626647 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1424/2878] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 23:00:41 2022] POST /identity/v3/users => generated 290 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:00:41.630501 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90de8034-523f-49de-8c78-9375304fcd49 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:41.641970 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90de8034-523f-49de-8c78-9375304fcd49 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:41.646646 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80aee0c-aeeb-4767-85a9-6a7ddc7d7ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:41.652414 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80aee0c-aeeb-4767-85a9-6a7ddc7d7ef1 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:41.652414 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80aee0c-aeeb-4767-85a9-6a7ddc7d7ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:41.652414 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80aee0c-aeeb-4767-85a9-6a7ddc7d7ef1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:41.652414 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80aee0c-aeeb-4767-85a9-6a7ddc7d7ef1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:41.652414 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80aee0c-aeeb-4767-85a9-6a7ddc7d7ef1 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:41.661666 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1455/2879] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 23:00:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:00:41.665654 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34fd9e3-2a59-412d-b7a3-ee5eb802ff51 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:41.676230 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34fd9e3-2a59-412d-b7a3-ee5eb802ff51 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:41.680905 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ba315-5587-46c5-b710-29eb8616e056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:41.681595 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ba315-5587-46c5-b710-29eb8616e056 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:41.681961 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ba315-5587-46c5-b710-29eb8616e056 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:41.682266 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ba315-5587-46c5-b710-29eb8616e056 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:41.682911 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ba315-5587-46c5-b710-29eb8616e056 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:41.684507 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ba315-5587-46c5-b710-29eb8616e056 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:41.694499 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1425/2880] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 23:00:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:00:41.699030 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80aee0c-aeeb-4767-85a9-6a7ddc7d7ef1 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:41.709083 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80aee0c-aeeb-4767-85a9-6a7ddc7d7ef1 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:41.714262 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f640e5-2047-4dc2-963f-46452f4554c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:41.714903 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f640e5-2047-4dc2-963f-46452f4554c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:41.715176 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f640e5-2047-4dc2-963f-46452f4554c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:41.715445 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f640e5-2047-4dc2-963f-46452f4554c7 admin admin] PATH_INFO: `/v3/projects/c0a016a3734c4590925ff9835c5780d0/users/6f4b7f13b8cb49f0a0c24b79ff18f0a1/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:41.730435 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f640e5-2047-4dc2-963f-46452f4554c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0a016a3734c4590925ff9835c5780d0, user_id=6f4b7f13b8cb49f0a0c24b79ff18f0a1, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:41.733109 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f640e5-2047-4dc2-963f-46452f4554c7 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:41.866594 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1456/2881] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 23:00:41 2022] PUT /identity/v3/projects/c0a016a3734c4590925ff9835c5780d0/users/6f4b7f13b8cb49f0a0c24b79ff18f0a1/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:00:41.879269 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd17dc53-3fb9-46c5-9fd9-0e3c292eef02 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:41.879469 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd17dc53-3fb9-46c5-9fd9-0e3c292eef02 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:41.879777 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd17dc53-3fb9-46c5-9fd9-0e3c292eef02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:41.908151 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-fd17dc53-3fb9-46c5-9fd9-0e3c292eef02 None None] MFA Rules not processed for user `6f4b7f13b8cb49f0a0c24b79ff18f0a1`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:00:41.909355 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-fd17dc53-3fb9-46c5-9fd9-0e3c292eef02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:41.973474 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1426/2882] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 23:00:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:00:41.977973 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f640e5-2047-4dc2-963f-46452f4554c7 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:42.031102 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:00:42.041993 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f640e5-2047-4dc2-963f-46452f4554c7 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:42.047112 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d6feb9-d1e4-4e7a-8128-d2c3505109fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:42.047753 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6feb9-d1e4-4e7a-8128-d2c3505109fc admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:42.048160 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6feb9-d1e4-4e7a-8128-d2c3505109fc admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:42.048442 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6feb9-d1e4-4e7a-8128-d2c3505109fc admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:42.049122 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d6feb9-d1e4-4e7a-8128-d2c3505109fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:42.051170 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d6feb9-d1e4-4e7a-8128-d2c3505109fc admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:42.173059 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1457/2883] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 23:00:41 2022] POST /identity/v3/projects => generated 377 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:00:42.177291 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd17dc53-3fb9-46c5-9fd9-0e3c292eef02 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:42.188374 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd17dc53-3fb9-46c5-9fd9-0e3c292eef02 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:42.215544 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:00:42.244426 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48461429-d400-42d7-be21-82287dc61b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:42.245056 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48461429-d400-42d7-be21-82287dc61b4a admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:42.245337 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48461429-d400-42d7-be21-82287dc61b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:42.245610 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48461429-d400-42d7-be21-82287dc61b4a admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:42.246248 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48461429-d400-42d7-be21-82287dc61b4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:42.248165 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48461429-d400-42d7-be21-82287dc61b4a admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:42.343332 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1427/2884] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 23:00:42 2022] POST /identity/v3/users => generated 295 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:00:42.347244 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d6feb9-d1e4-4e7a-8128-d2c3505109fc admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:42.358812 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d6feb9-d1e4-4e7a-8128-d2c3505109fc admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:42.364054 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6a8ec-fda2-447a-994c-a1d11d10dab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:42.364662 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6a8ec-fda2-447a-994c-a1d11d10dab9 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:42.364976 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6a8ec-fda2-447a-994c-a1d11d10dab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:42.365252 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6a8ec-fda2-447a-994c-a1d11d10dab9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:42.366062 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b6a8ec-fda2-447a-994c-a1d11d10dab9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:42.367790 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b6a8ec-fda2-447a-994c-a1d11d10dab9 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:42.378383 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1458/2885] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 23:00:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:00:42.382604 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48461429-d400-42d7-be21-82287dc61b4a admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:42.393905 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48461429-d400-42d7-be21-82287dc61b4a admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:42.398677 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0a6a4e-80fd-4d05-b578-f7b3c215747f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:42.400351 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a6a4e-80fd-4d05-b578-f7b3c215747f admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:42.400351 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a6a4e-80fd-4d05-b578-f7b3c215747f admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:42.400351 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a6a4e-80fd-4d05-b578-f7b3c215747f admin admin] PATH_INFO: `/v3/projects/47fed41afbe842d491c0c55adfed9477/users/a7a65c5d4d7a4a5ebd8c324e25621579/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:42.415465 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0a6a4e-80fd-4d05-b578-f7b3c215747f admin admin] RBAC: Authorizing `identity:create_grant(project_id=47fed41afbe842d491c0c55adfed9477, user_id=a7a65c5d4d7a4a5ebd8c324e25621579, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:42.419166 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0a6a4e-80fd-4d05-b578-f7b3c215747f admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:42.474258 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1428/2886] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 23:00:42 2022] PUT /identity/v3/projects/47fed41afbe842d491c0c55adfed9477/users/a7a65c5d4d7a4a5ebd8c324e25621579/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:00:42.478635 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6a8ec-fda2-447a-994c-a1d11d10dab9 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:42.489927 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6a8ec-fda2-447a-994c-a1d11d10dab9 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:42.553009 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b072c21-c07b-49ea-85de-16272f16047d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:42.553763 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b072c21-c07b-49ea-85de-16272f16047d admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:42.554145 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b072c21-c07b-49ea-85de-16272f16047d admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:42.554521 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b072c21-c07b-49ea-85de-16272f16047d admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:42.555268 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b072c21-c07b-49ea-85de-16272f16047d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:42.557979 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b072c21-c07b-49ea-85de-16272f16047d admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:42.568822 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1459/2887] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 23:00:42 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:00:42.574235 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0a6a4e-80fd-4d05-b578-f7b3c215747f admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:42.585142 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0a6a4e-80fd-4d05-b578-f7b3c215747f admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:42.590459 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f1f699-1f17-49f0-91bf-5c1368ee07ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:42.591113 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f1f699-1f17-49f0-91bf-5c1368ee07ba admin admin] REQUEST_METHOD: `PUT` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:42.591394 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f1f699-1f17-49f0-91bf-5c1368ee07ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:42.591671 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f1f699-1f17-49f0-91bf-5c1368ee07ba admin admin] PATH_INFO: `/v3/projects/47fed41afbe842d491c0c55adfed9477/users/a7a65c5d4d7a4a5ebd8c324e25621579/roles/882f35bd847a406eb2222aeafc517a46` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:42.593866 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f1f699-1f17-49f0-91bf-5c1368ee07ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=47fed41afbe842d491c0c55adfed9477, user_id=a7a65c5d4d7a4a5ebd8c324e25621579, role_id=882f35bd847a406eb2222aeafc517a46)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:42.596509 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f1f699-1f17-49f0-91bf-5c1368ee07ba admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:42.607471 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1429/2888] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 23:00:42 2022] PUT /identity/v3/projects/47fed41afbe842d491c0c55adfed9477/users/a7a65c5d4d7a4a5ebd8c324e25621579/roles/882f35bd847a406eb2222aeafc517a46 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:00:42.617788 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7865665-fa73-4f58-a207-73f64e67658a None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:42.618133 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7865665-fa73-4f58-a207-73f64e67658a None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:42.618342 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7865665-fa73-4f58-a207-73f64e67658a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:42.647695 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-e7865665-fa73-4f58-a207-73f64e67658a None None] MFA Rules not processed for user `a7a65c5d4d7a4a5ebd8c324e25621579`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:00:42.648941 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-e7865665-fa73-4f58-a207-73f64e67658a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:42.717700 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1460/2889] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 23:00:42 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:00:42.729093 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f1f699-1f17-49f0-91bf-5c1368ee07ba admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:42.730854 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-49f1f699-1f17-49f0-91bf-5c1368ee07ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:42.789687 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f1f699-1f17-49f0-91bf-5c1368ee07ba admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:42.793439 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859d9b3a-1e09-48dd-9eaa-1a34c66d790c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:42.794038 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d9b3a-1e09-48dd-9eaa-1a34c66d790c service cinder] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:42.794314 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d9b3a-1e09-48dd-9eaa-1a34c66d790c service cinder] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:42.794581 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d9b3a-1e09-48dd-9eaa-1a34c66d790c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:42.803437 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859d9b3a-1e09-48dd-9eaa-1a34c66d790c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:42.804891 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859d9b3a-1e09-48dd-9eaa-1a34c66d790c service cinder] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:42.814663 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1430/2890] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 23:00:42 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:00:42.917787 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7865665-fa73-4f58-a207-73f64e67658a None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:42.919204 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-e7865665-fa73-4f58-a207-73f64e67658a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:42.931126 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7865665-fa73-4f58-a207-73f64e67658a None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:42.935232 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c998566-ebad-47b8-a44e-2d76e92c85c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:42.935825 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c998566-ebad-47b8-a44e-2d76e92c85c0 service cinder] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:42.936106 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c998566-ebad-47b8-a44e-2d76e92c85c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:42.936376 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c998566-ebad-47b8-a44e-2d76e92c85c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:42.982673 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:00:42.995177 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c998566-ebad-47b8-a44e-2d76e92c85c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:42.996588 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c998566-ebad-47b8-a44e-2d76e92c85c0 service cinder] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:43.005346 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1461/2891] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 23:00:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:00:47.049028 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859d9b3a-1e09-48dd-9eaa-1a34c66d790c service cinder] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:47.112112 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859d9b3a-1e09-48dd-9eaa-1a34c66d790c service cinder] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:47.116612 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf01aef-cddc-40c1-aaa8-79a29320a63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:47.117262 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf01aef-cddc-40c1-aaa8-79a29320a63b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:47.117542 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf01aef-cddc-40c1-aaa8-79a29320a63b admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:47.117813 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf01aef-cddc-40c1-aaa8-79a29320a63b admin admin] PATH_INFO: `/v3/users/a53a9eae01064be99a9fda1f50b62358` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:47.118917 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf01aef-cddc-40c1-aaa8-79a29320a63b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a53a9eae01064be99a9fda1f50b62358)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:47.120843 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf01aef-cddc-40c1-aaa8-79a29320a63b admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:47.204528 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:00:47.305377 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-7bf01aef-cddc-40c1-aaa8-79a29320a63b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a53a9eae01064be99a9fda1f50b62358'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:47.317334 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-7bf01aef-cddc-40c1-aaa8-79a29320a63b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a53a9eae01064be99a9fda1f50b62358'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:47.323949 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-7bf01aef-cddc-40c1-aaa8-79a29320a63b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a53a9eae01064be99a9fda1f50b62358'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:47.405234 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-7bf01aef-cddc-40c1-aaa8-79a29320a63b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a53a9eae01064be99a9fda1f50b62358'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:47.405852 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-7bf01aef-cddc-40c1-aaa8-79a29320a63b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a53a9eae01064be99a9fda1f50b62358'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:47.412061 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1431/2892] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 23:00:47 2022] DELETE /identity/v3/users/a53a9eae01064be99a9fda1f50b62358 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:00:47.416807 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d32936-b21d-4ce8-9327-8d7896e6a3f6 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:47.417299 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d32936-b21d-4ce8-9327-8d7896e6a3f6 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:47.417412 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d32936-b21d-4ce8-9327-8d7896e6a3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:47.436646 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-b8d32936-b21d-4ce8-9327-8d7896e6a3f6 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:00:47.439746 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-b8d32936-b21d-4ce8-9327-8d7896e6a3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:47.502965 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1462/2893] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 23:00:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:00:47.513590 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf01aef-cddc-40c1-aaa8-79a29320a63b admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:47.515019 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-7bf01aef-cddc-40c1-aaa8-79a29320a63b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:47.572228 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf01aef-cddc-40c1-aaa8-79a29320a63b admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:47.576482 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc06623a-b674-4027-83c3-c657a4914ae8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:47.577118 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc06623a-b674-4027-83c3-c657a4914ae8 service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:47.577399 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc06623a-b674-4027-83c3-c657a4914ae8 service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:47.577673 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc06623a-b674-4027-83c3-c657a4914ae8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:47.586654 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc06623a-b674-4027-83c3-c657a4914ae8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:47.588082 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc06623a-b674-4027-83c3-c657a4914ae8 service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:47.598220 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1432/2894] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 23:00:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:00:47.978378 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d32936-b21d-4ce8-9327-8d7896e6a3f6 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:47.978378 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-b8d32936-b21d-4ce8-9327-8d7896e6a3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:47.992197 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d32936-b21d-4ce8-9327-8d7896e6a3f6 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:47.997395 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ea142a-6223-46b0-8f5b-ff0beb714314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:47.998658 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ea142a-6223-46b0-8f5b-ff0beb714314 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:47.998945 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ea142a-6223-46b0-8f5b-ff0beb714314 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:47.999218 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ea142a-6223-46b0-8f5b-ff0beb714314 admin admin] PATH_INFO: `/v3/projects/004855483053406bbe809eddf288a559` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:48.000524 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ea142a-6223-46b0-8f5b-ff0beb714314 admin admin] RBAC: Authorizing `identity:delete_project(project_id=004855483053406bbe809eddf288a559)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:48.002576 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ea142a-6223-46b0-8f5b-ff0beb714314 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:48.112789 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d7ea142a-6223-46b0-8f5b-ff0beb714314 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '004855483053406bbe809eddf288a559', 'request_id': 'req-d7ea142a-6223-46b0-8f5b-ff0beb714314'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:48.118218 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d7ea142a-6223-46b0-8f5b-ff0beb714314 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '004855483053406bbe809eddf288a559', 'request_id': 'req-d7ea142a-6223-46b0-8f5b-ff0beb714314'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:48.125011 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d7ea142a-6223-46b0-8f5b-ff0beb714314 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '004855483053406bbe809eddf288a559', 'request_id': 'req-d7ea142a-6223-46b0-8f5b-ff0beb714314'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:48.197426 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d7ea142a-6223-46b0-8f5b-ff0beb714314 admin admin] The token cache is being invalidate because project 004855483053406bbe809eddf288a559 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:00:48.197718 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d7ea142a-6223-46b0-8f5b-ff0beb714314 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:48.198370 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-d7ea142a-6223-46b0-8f5b-ff0beb714314 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:48.199898 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1463/2895] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 23:00:47 2022] DELETE /identity/v3/projects/004855483053406bbe809eddf288a559 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:00:48.203878 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc06623a-b674-4027-83c3-c657a4914ae8 service neutron] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:48.214137 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc06623a-b674-4027-83c3-c657a4914ae8 service neutron] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:48.272424 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ffd30f-9836-498f-8702-17cab3607815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:48.273089 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ffd30f-9836-498f-8702-17cab3607815 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:48.273383 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ffd30f-9836-498f-8702-17cab3607815 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:48.273662 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ffd30f-9836-498f-8702-17cab3607815 admin admin] PATH_INFO: `/v3/users/6f4b7f13b8cb49f0a0c24b79ff18f0a1` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:48.274799 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ffd30f-9836-498f-8702-17cab3607815 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f4b7f13b8cb49f0a0c24b79ff18f0a1)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:48.276840 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ffd30f-9836-498f-8702-17cab3607815 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:48.475810 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-b4ffd30f-9836-498f-8702-17cab3607815 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f4b7f13b8cb49f0a0c24b79ff18f0a1'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:48.476940 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:00:48.493649 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-b4ffd30f-9836-498f-8702-17cab3607815 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f4b7f13b8cb49f0a0c24b79ff18f0a1'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:48.502085 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-b4ffd30f-9836-498f-8702-17cab3607815 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f4b7f13b8cb49f0a0c24b79ff18f0a1'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:48.586420 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-b4ffd30f-9836-498f-8702-17cab3607815 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f4b7f13b8cb49f0a0c24b79ff18f0a1'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:48.587062 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-b4ffd30f-9836-498f-8702-17cab3607815 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f4b7f13b8cb49f0a0c24b79ff18f0a1'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:48.594168 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1433/2896] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 23:00:48 2022] DELETE /identity/v3/users/6f4b7f13b8cb49f0a0c24b79ff18f0a1 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:00:49.001248 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ea142a-6223-46b0-8f5b-ff0beb714314 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:49.002704 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-d7ea142a-6223-46b0-8f5b-ff0beb714314 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:49.018355 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ea142a-6223-46b0-8f5b-ff0beb714314 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:49.078004 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ded48d-45ee-4188-9a30-4446d8f3b12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:49.078636 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ded48d-45ee-4188-9a30-4446d8f3b12a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:49.078920 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ded48d-45ee-4188-9a30-4446d8f3b12a admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:49.079259 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ded48d-45ee-4188-9a30-4446d8f3b12a admin admin] PATH_INFO: `/v3/projects/c0a016a3734c4590925ff9835c5780d0` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:49.080316 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ded48d-45ee-4188-9a30-4446d8f3b12a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0a016a3734c4590925ff9835c5780d0)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:49.082270 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ded48d-45ee-4188-9a30-4446d8f3b12a admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:49.094437 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:00:49.184696 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-99ded48d-45ee-4188-9a30-4446d8f3b12a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0a016a3734c4590925ff9835c5780d0', 'request_id': 'req-99ded48d-45ee-4188-9a30-4446d8f3b12a'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:49.190201 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-99ded48d-45ee-4188-9a30-4446d8f3b12a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0a016a3734c4590925ff9835c5780d0', 'request_id': 'req-99ded48d-45ee-4188-9a30-4446d8f3b12a'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:49.196595 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-99ded48d-45ee-4188-9a30-4446d8f3b12a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0a016a3734c4590925ff9835c5780d0', 'request_id': 'req-99ded48d-45ee-4188-9a30-4446d8f3b12a'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:49.279702 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-99ded48d-45ee-4188-9a30-4446d8f3b12a admin admin] The token cache is being invalidate because project c0a016a3734c4590925ff9835c5780d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:00:49.279981 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-99ded48d-45ee-4188-9a30-4446d8f3b12a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:49.280537 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-99ded48d-45ee-4188-9a30-4446d8f3b12a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:49.282075 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1464/2897] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 23:00:49 2022] DELETE /identity/v3/projects/c0a016a3734c4590925ff9835c5780d0 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:00:49.286829 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ffd30f-9836-498f-8702-17cab3607815 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:49.287580 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-b4ffd30f-9836-498f-8702-17cab3607815 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:49.300643 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ffd30f-9836-498f-8702-17cab3607815 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:49.358479 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf72d132-343f-4b12-9eaf-9907b50291eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:49.359115 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72d132-343f-4b12-9eaf-9907b50291eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:49.359403 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72d132-343f-4b12-9eaf-9907b50291eb admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:49.359677 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72d132-343f-4b12-9eaf-9907b50291eb admin admin] PATH_INFO: `/v3/users/a7a65c5d4d7a4a5ebd8c324e25621579` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:49.360825 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf72d132-343f-4b12-9eaf-9907b50291eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7a65c5d4d7a4a5ebd8c324e25621579)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:49.362749 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf72d132-343f-4b12-9eaf-9907b50291eb admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:49.531679 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-bf72d132-343f-4b12-9eaf-9907b50291eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7a65c5d4d7a4a5ebd8c324e25621579'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:49.544632 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-bf72d132-343f-4b12-9eaf-9907b50291eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7a65c5d4d7a4a5ebd8c324e25621579'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:49.551739 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-bf72d132-343f-4b12-9eaf-9907b50291eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7a65c5d4d7a4a5ebd8c324e25621579'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:49.638538 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-bf72d132-343f-4b12-9eaf-9907b50291eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7a65c5d4d7a4a5ebd8c324e25621579'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:49.642210 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-bf72d132-343f-4b12-9eaf-9907b50291eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7a65c5d4d7a4a5ebd8c324e25621579'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:49.646172 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1434/2898] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 23:00:49 2022] DELETE /identity/v3/users/a7a65c5d4d7a4a5ebd8c324e25621579 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:00:50.302726 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ded48d-45ee-4188-9a30-4446d8f3b12a admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:50.304219 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-99ded48d-45ee-4188-9a30-4446d8f3b12a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:50.318823 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ded48d-45ee-4188-9a30-4446d8f3b12a admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:50.377308 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb76d78-6b16-4d4b-8900-609b808eaee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:50.377869 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb76d78-6b16-4d4b-8900-609b808eaee5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:50.378150 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb76d78-6b16-4d4b-8900-609b808eaee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:50.378425 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb76d78-6b16-4d4b-8900-609b808eaee5 admin admin] PATH_INFO: `/v3/projects/47fed41afbe842d491c0c55adfed9477` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:50.379640 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb76d78-6b16-4d4b-8900-609b808eaee5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47fed41afbe842d491c0c55adfed9477)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:50.381657 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb76d78-6b16-4d4b-8900-609b808eaee5 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:50.474045 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-1bb76d78-6b16-4d4b-8900-609b808eaee5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47fed41afbe842d491c0c55adfed9477', 'request_id': 'req-1bb76d78-6b16-4d4b-8900-609b808eaee5'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:50.479771 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-1bb76d78-6b16-4d4b-8900-609b808eaee5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47fed41afbe842d491c0c55adfed9477', 'request_id': 'req-1bb76d78-6b16-4d4b-8900-609b808eaee5'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:50.486970 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-1bb76d78-6b16-4d4b-8900-609b808eaee5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47fed41afbe842d491c0c55adfed9477', 'request_id': 'req-1bb76d78-6b16-4d4b-8900-609b808eaee5'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:50.548382 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-1bb76d78-6b16-4d4b-8900-609b808eaee5 admin admin] The token cache is being invalidate because project 47fed41afbe842d491c0c55adfed9477 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:00:50.553446 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-1bb76d78-6b16-4d4b-8900-609b808eaee5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:50.553446 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-1bb76d78-6b16-4d4b-8900-609b808eaee5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:50.553446 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1465/2899] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 23:00:50 2022] DELETE /identity/v3/projects/47fed41afbe842d491c0c55adfed9477 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:00:50.556737 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32189650-5713-4a70-b49e-ee16f8215c52 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:50.557215 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32189650-5713-4a70-b49e-ee16f8215c52 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:50.557337 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32189650-5713-4a70-b49e-ee16f8215c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:50.578031 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-32189650-5713-4a70-b49e-ee16f8215c52 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:00:50.582014 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-32189650-5713-4a70-b49e-ee16f8215c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:50.619837 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:00:50.638983 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1435/2900] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 23:00:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:00:50.648663 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8414d24-a344-4476-80ff-61a4007fe674 None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:50.648980 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8414d24-a344-4476-80ff-61a4007fe674 None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:50.649322 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8414d24-a344-4476-80ff-61a4007fe674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:50.669675 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-f8414d24-a344-4476-80ff-61a4007fe674 None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:00:50.679254 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-f8414d24-a344-4476-80ff-61a4007fe674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:50.679254 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1466/2901] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 23:00:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:00:50.683162 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32189650-5713-4a70-b49e-ee16f8215c52 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:50.694345 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32189650-5713-4a70-b49e-ee16f8215c52 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:50.699556 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0a48e5-4d8e-4c4e-a14f-ac1275cc9818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:50.702222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a48e5-4d8e-4c4e-a14f-ac1275cc9818 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:50.702222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a48e5-4d8e-4c4e-a14f-ac1275cc9818 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:50.702222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a48e5-4d8e-4c4e-a14f-ac1275cc9818 admin admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:50.702222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0a48e5-4d8e-4c4e-a14f-ac1275cc9818 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:00:50.702222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0a48e5-4d8e-4c4e-a14f-ac1275cc9818 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:50.703643 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0a48e5-4d8e-4c4e-a14f-ac1275cc9818 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:50.715401 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1436/2902] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 23:00:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:00:50.719913 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8414d24-a344-4476-80ff-61a4007fe674 None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:50.729372 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8414d24-a344-4476-80ff-61a4007fe674 None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:50.734668 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea1a03-85fe-4bb8-a8fa-02f27cc3665d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:50.735268 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea1a03-85fe-4bb8-a8fa-02f27cc3665d admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:50.735581 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea1a03-85fe-4bb8-a8fa-02f27cc3665d admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:50.735855 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea1a03-85fe-4bb8-a8fa-02f27cc3665d admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:50.736480 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aea1a03-85fe-4bb8-a8fa-02f27cc3665d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:50.738414 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aea1a03-85fe-4bb8-a8fa-02f27cc3665d admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:50.836165 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1467/2903] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 23:00:50 2022] POST /identity/v3/projects => generated 375 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:00:50.840160 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0a48e5-4d8e-4c4e-a14f-ac1275cc9818 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:50.851130 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0a48e5-4d8e-4c4e-a14f-ac1275cc9818 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:50.907712 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c62bbb5-866f-4940-afe6-80555bb2fec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:50.908248 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c62bbb5-866f-4940-afe6-80555bb2fec6 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:50.908529 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c62bbb5-866f-4940-afe6-80555bb2fec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:50.908836 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c62bbb5-866f-4940-afe6-80555bb2fec6 admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:50.909472 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c62bbb5-866f-4940-afe6-80555bb2fec6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:50.911394 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c62bbb5-866f-4940-afe6-80555bb2fec6 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:51.000036 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1437/2904] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 23:00:50 2022] POST /identity/v3/users => generated 288 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:00:51.004499 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea1a03-85fe-4bb8-a8fa-02f27cc3665d admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:51.010005 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:00:51.015735 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea1a03-85fe-4bb8-a8fa-02f27cc3665d admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:51.020477 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1b5e18-a54e-4593-9e49-3b88e9be7e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:51.021116 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b5e18-a54e-4593-9e49-3b88e9be7e00 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:51.021396 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b5e18-a54e-4593-9e49-3b88e9be7e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:51.021669 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b5e18-a54e-4593-9e49-3b88e9be7e00 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:51.022310 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1b5e18-a54e-4593-9e49-3b88e9be7e00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:51.023939 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1b5e18-a54e-4593-9e49-3b88e9be7e00 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:51.034837 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1468/2905] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 23:00:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:00:51.038795 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c62bbb5-866f-4940-afe6-80555bb2fec6 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:51.049311 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c62bbb5-866f-4940-afe6-80555bb2fec6 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:51.054229 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c832e6-be7f-46a9-ab10-e3b74c0026e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:51.054816 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c832e6-be7f-46a9-ab10-e3b74c0026e9 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:51.055092 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c832e6-be7f-46a9-ab10-e3b74c0026e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:51.055412 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c832e6-be7f-46a9-ab10-e3b74c0026e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:51.056162 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c832e6-be7f-46a9-ab10-e3b74c0026e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:51.057770 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c832e6-be7f-46a9-ab10-e3b74c0026e9 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:51.068978 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1438/2906] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 23:00:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:00:51.073194 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1b5e18-a54e-4593-9e49-3b88e9be7e00 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:51.083767 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1b5e18-a54e-4593-9e49-3b88e9be7e00 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:51.088519 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd9172d-2aef-4f9f-ae9c-64c6aadbb91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:51.089194 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9172d-2aef-4f9f-ae9c-64c6aadbb91d admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:51.089476 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9172d-2aef-4f9f-ae9c-64c6aadbb91d admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:51.089748 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9172d-2aef-4f9f-ae9c-64c6aadbb91d admin admin] PATH_INFO: `/v3/projects/3b365071ff2148beab7a8e3fea894f0d/users/727402cbb75b48389bcab8700cee38f1/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:51.105002 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd9172d-2aef-4f9f-ae9c-64c6aadbb91d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b365071ff2148beab7a8e3fea894f0d, user_id=727402cbb75b48389bcab8700cee38f1, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:51.108541 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd9172d-2aef-4f9f-ae9c-64c6aadbb91d admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:51.240144 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1469/2907] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 23:00:51 2022] PUT /identity/v3/projects/3b365071ff2148beab7a8e3fea894f0d/users/727402cbb75b48389bcab8700cee38f1/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:00:51.251815 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b7a4c-60f5-4251-b2c2-510b05315026 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:51.252103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b7a4c-60f5-4251-b2c2-510b05315026 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:51.252380 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b7a4c-60f5-4251-b2c2-510b05315026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:51.287520 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-098b7a4c-60f5-4251-b2c2-510b05315026 None None] MFA Rules not processed for user `727402cbb75b48389bcab8700cee38f1`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:00:51.288746 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-098b7a4c-60f5-4251-b2c2-510b05315026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:51.353413 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1439/2908] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 23:00:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:00:51.365063 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd9172d-2aef-4f9f-ae9c-64c6aadbb91d admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:51.366810 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-ecd9172d-2aef-4f9f-ae9c-64c6aadbb91d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:51.423218 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd9172d-2aef-4f9f-ae9c-64c6aadbb91d admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:51.426911 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999ca65-7ef3-4cf9-b140-17f5120edc40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:51.427505 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999ca65-7ef3-4cf9-b140-17f5120edc40 service cinder] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:51.427783 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999ca65-7ef3-4cf9-b140-17f5120edc40 service cinder] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:51.428053 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999ca65-7ef3-4cf9-b140-17f5120edc40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:51.436841 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9999ca65-7ef3-4cf9-b140-17f5120edc40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:51.438544 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9999ca65-7ef3-4cf9-b140-17f5120edc40 service cinder] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:51.447466 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1470/2909] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 23:00:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:00:55.391298 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b7a4c-60f5-4251-b2c2-510b05315026 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:55.433471 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:00:55.454608 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b7a4c-60f5-4251-b2c2-510b05315026 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:55.459771 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a6e5f-5e66-4d19-a158-feab1e311716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:55.460396 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a6e5f-5e66-4d19-a158-feab1e311716 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:55.460678 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a6e5f-5e66-4d19-a158-feab1e311716 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:55.460984 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a6e5f-5e66-4d19-a158-feab1e311716 admin admin] PATH_INFO: `/v3/users/727402cbb75b48389bcab8700cee38f1` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:55.462367 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0a6e5f-5e66-4d19-a158-feab1e311716 admin admin] RBAC: Authorizing `identity:delete_user(user_id=727402cbb75b48389bcab8700cee38f1)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:55.464742 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0a6e5f-5e66-4d19-a158-feab1e311716 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:55.682431 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-2f0a6e5f-5e66-4d19-a158-feab1e311716 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '727402cbb75b48389bcab8700cee38f1'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:55.695125 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-2f0a6e5f-5e66-4d19-a158-feab1e311716 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '727402cbb75b48389bcab8700cee38f1'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:55.702981 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-2f0a6e5f-5e66-4d19-a158-feab1e311716 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '727402cbb75b48389bcab8700cee38f1'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:55.753734 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-2f0a6e5f-5e66-4d19-a158-feab1e311716 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '727402cbb75b48389bcab8700cee38f1'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:55.754359 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-2f0a6e5f-5e66-4d19-a158-feab1e311716 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '727402cbb75b48389bcab8700cee38f1'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:55.760651 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1440/2910] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 23:00:55 2022] DELETE /identity/v3/users/727402cbb75b48389bcab8700cee38f1 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:00:55.766047 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496a0b4-46ab-42ee-9a1a-2b2a42c3b28b None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:55.766589 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496a0b4-46ab-42ee-9a1a-2b2a42c3b28b None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:55.766589 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496a0b4-46ab-42ee-9a1a-2b2a42c3b28b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:55.786356 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-0496a0b4-46ab-42ee-9a1a-2b2a42c3b28b None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:00:55.787592 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-0496a0b4-46ab-42ee-9a1a-2b2a42c3b28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:55.847443 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1471/2911] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 23:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:00:55.857272 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a6e5f-5e66-4d19-a158-feab1e311716 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:55.858765 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-2f0a6e5f-5e66-4d19-a158-feab1e311716 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:55.914751 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a6e5f-5e66-4d19-a158-feab1e311716 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:55.918278 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6754dd-2077-4ec8-a6fa-6bf0463bb9fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:55.918866 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6754dd-2077-4ec8-a6fa-6bf0463bb9fd service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:55.919142 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6754dd-2077-4ec8-a6fa-6bf0463bb9fd service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:55.919412 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6754dd-2077-4ec8-a6fa-6bf0463bb9fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:55.928487 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6754dd-2077-4ec8-a6fa-6bf0463bb9fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:55.929935 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6754dd-2077-4ec8-a6fa-6bf0463bb9fd service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:55.939754 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1441/2912] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 23:00:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:00:56.347873 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0496a0b4-46ab-42ee-9a1a-2b2a42c3b28b None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:56.349338 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-0496a0b4-46ab-42ee-9a1a-2b2a42c3b28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:56.362291 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0496a0b4-46ab-42ee-9a1a-2b2a42c3b28b None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:56.367036 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b63ec-acdb-48d5-a3fc-6a7b5428c633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:56.367663 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b63ec-acdb-48d5-a3fc-6a7b5428c633 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:56.367940 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b63ec-acdb-48d5-a3fc-6a7b5428c633 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:56.368208 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b63ec-acdb-48d5-a3fc-6a7b5428c633 admin admin] PATH_INFO: `/v3/projects/3b365071ff2148beab7a8e3fea894f0d` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:56.369383 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8b63ec-acdb-48d5-a3fc-6a7b5428c633 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b365071ff2148beab7a8e3fea894f0d)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:56.371278 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8b63ec-acdb-48d5-a3fc-6a7b5428c633 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:56.470109 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ae8b63ec-acdb-48d5-a3fc-6a7b5428c633 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b365071ff2148beab7a8e3fea894f0d', 'request_id': 'req-ae8b63ec-acdb-48d5-a3fc-6a7b5428c633'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:56.475582 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ae8b63ec-acdb-48d5-a3fc-6a7b5428c633 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b365071ff2148beab7a8e3fea894f0d', 'request_id': 'req-ae8b63ec-acdb-48d5-a3fc-6a7b5428c633'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:56.481734 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ae8b63ec-acdb-48d5-a3fc-6a7b5428c633 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b365071ff2148beab7a8e3fea894f0d', 'request_id': 'req-ae8b63ec-acdb-48d5-a3fc-6a7b5428c633'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:56.484229 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:00:56.551411 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ae8b63ec-acdb-48d5-a3fc-6a7b5428c633 admin admin] The token cache is being invalidate because project 3b365071ff2148beab7a8e3fea894f0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:00:56.551705 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ae8b63ec-acdb-48d5-a3fc-6a7b5428c633 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:56.552340 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ae8b63ec-acdb-48d5-a3fc-6a7b5428c633 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:00:56.553868 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1472/2913] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 23:00:56 2022] DELETE /identity/v3/projects/3b365071ff2148beab7a8e3fea894f0d => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:00:56.558664 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1cfe71-5b23-41c5-96dc-9944d75cd3da None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:56.559165 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1cfe71-5b23-41c5-96dc-9944d75cd3da None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:56.559286 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1cfe71-5b23-41c5-96dc-9944d75cd3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:56.587583 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-4b1cfe71-5b23-41c5-96dc-9944d75cd3da None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:00:56.588840 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-4b1cfe71-5b23-41c5-96dc-9944d75cd3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:56.777397 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1442/2914] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 23:00:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:00:56.785781 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21456a6b-58ef-4c77-8516-b0759eb69fea None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:56.786317 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21456a6b-58ef-4c77-8516-b0759eb69fea None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:56.786317 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21456a6b-58ef-4c77-8516-b0759eb69fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:56.806416 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-21456a6b-58ef-4c77-8516-b0759eb69fea None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:00:56.807629 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-21456a6b-58ef-4c77-8516-b0759eb69fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:56.816185 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1473/2915] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 23:00:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:00:56.820400 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1cfe71-5b23-41c5-96dc-9944d75cd3da None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:56.825168 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:00:56.830686 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1cfe71-5b23-41c5-96dc-9944d75cd3da None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:56.835728 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f355f-9cb6-466e-847a-2284360d4660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:56.836328 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f355f-9cb6-466e-847a-2284360d4660 admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:56.836629 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f355f-9cb6-466e-847a-2284360d4660 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:56.836949 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f355f-9cb6-466e-847a-2284360d4660 admin admin] PATH_INFO: `/v3/domains` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:56.837594 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3f355f-9cb6-466e-847a-2284360d4660 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:00:56.837897 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3f355f-9cb6-466e-847a-2284360d4660 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:56.839485 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3f355f-9cb6-466e-847a-2284360d4660 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:56.851608 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1443/2916] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Aug 29 23:00:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:00:56.856056 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21456a6b-58ef-4c77-8516-b0759eb69fea None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:56.866163 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21456a6b-58ef-4c77-8516-b0759eb69fea None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:56.871116 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddbaca8-2436-4644-8b45-5289e0a70126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:56.871705 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddbaca8-2436-4644-8b45-5289e0a70126 admin admin] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:56.871980 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddbaca8-2436-4644-8b45-5289e0a70126 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:56.872250 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddbaca8-2436-4644-8b45-5289e0a70126 admin admin] PATH_INFO: `/v3/projects` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:56.872924 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddbaca8-2436-4644-8b45-5289e0a70126 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:56.875680 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddbaca8-2436-4644-8b45-5289e0a70126 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:56.961403 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1474/2917] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Aug 29 23:00:56 2022] POST /identity/v3/projects => generated 371 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:00:56.965596 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f355f-9cb6-466e-847a-2284360d4660 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:56.975487 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f355f-9cb6-466e-847a-2284360d4660 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:57.030495 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6075a14c-3c47-4122-83a9-c1a525b56e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:57.030880 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6075a14c-3c47-4122-83a9-c1a525b56e69 admin admin] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:57.031159 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6075a14c-3c47-4122-83a9-c1a525b56e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:57.031430 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6075a14c-3c47-4122-83a9-c1a525b56e69 admin admin] PATH_INFO: `/v3/users` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:57.032058 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6075a14c-3c47-4122-83a9-c1a525b56e69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:57.033989 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6075a14c-3c47-4122-83a9-c1a525b56e69 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:57.136582 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1444/2918] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Aug 29 23:00:56 2022] POST /identity/v3/users => generated 286 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:00:57.141124 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddbaca8-2436-4644-8b45-5289e0a70126 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:57.151611 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddbaca8-2436-4644-8b45-5289e0a70126 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:57.156854 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06249a55-2ca6-404d-af04-1ef323999648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:57.157451 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06249a55-2ca6-404d-af04-1ef323999648 admin admin] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:57.157733 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06249a55-2ca6-404d-af04-1ef323999648 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:57.158010 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06249a55-2ca6-404d-af04-1ef323999648 admin admin] PATH_INFO: `/v3/roles` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:57.158656 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06249a55-2ca6-404d-af04-1ef323999648 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:57.160251 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06249a55-2ca6-404d-af04-1ef323999648 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:57.170315 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1475/2919] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 23:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:00:57.175027 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6075a14c-3c47-4122-83a9-c1a525b56e69 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:57.185090 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6075a14c-3c47-4122-83a9-c1a525b56e69 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:57.189844 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b8b00-cdd8-4b07-99a1-e596d19d89cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:57.190422 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b8b00-cdd8-4b07-99a1-e596d19d89cc admin admin] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:57.190695 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b8b00-cdd8-4b07-99a1-e596d19d89cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:57.190962 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b8b00-cdd8-4b07-99a1-e596d19d89cc admin admin] PATH_INFO: `/v3/roles` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:57.191589 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880b8b00-cdd8-4b07-99a1-e596d19d89cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:57.193258 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880b8b00-cdd8-4b07-99a1-e596d19d89cc admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:57.203528 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1445/2920] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Aug 29 23:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:00:57.207605 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06249a55-2ca6-404d-af04-1ef323999648 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:57.218586 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06249a55-2ca6-404d-af04-1ef323999648 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:57.223271 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f8f1de-da3d-42fa-bf4c-3813c2111c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:57.223921 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f8f1de-da3d-42fa-bf4c-3813c2111c30 admin admin] REQUEST_METHOD: `PUT` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:57.224204 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f8f1de-da3d-42fa-bf4c-3813c2111c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:57.224479 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f8f1de-da3d-42fa-bf4c-3813c2111c30 admin admin] PATH_INFO: `/v3/projects/260ffd54bf93432691ee9bdc1bf96139/users/bcd7a111086c4564bd86c09b7c859358/roles/ae06bb653e3b4990a9ddaa1a649c7e06` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:57.240391 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f8f1de-da3d-42fa-bf4c-3813c2111c30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=260ffd54bf93432691ee9bdc1bf96139, user_id=bcd7a111086c4564bd86c09b7c859358, role_id=ae06bb653e3b4990a9ddaa1a649c7e06)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:57.243127 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f8f1de-da3d-42fa-bf4c-3813c2111c30 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:57.438296 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1476/2921] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Aug 29 23:00:57 2022] PUT /identity/v3/projects/260ffd54bf93432691ee9bdc1bf96139/users/bcd7a111086c4564bd86c09b7c859358/roles/ae06bb653e3b4990a9ddaa1a649c7e06 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:00:57.449563 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da907903-7f39-4ba5-ae13-fc2ec1c5a625 None None] REQUEST_METHOD: `POST` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:57.449872 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da907903-7f39-4ba5-ae13-fc2ec1c5a625 None None] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:57.450228 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da907903-7f39-4ba5-ae13-fc2ec1c5a625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:57.477772 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.auth.core [None req-da907903-7f39-4ba5-ae13-fc2ec1c5a625 None None] MFA Rules not processed for user `bcd7a111086c4564bd86c09b7c859358`. Rule list: `[]` (Enabled: `True`). {{(pid=93272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:00:57.479130 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-da907903-7f39-4ba5-ae13-fc2ec1c5a625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:57.547104 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1446/2922] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 23:00:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:00:57.557667 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f8f1de-da3d-42fa-bf4c-3813c2111c30 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:57.559342 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-e2f8f1de-da3d-42fa-bf4c-3813c2111c30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:57.618017 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f8f1de-da3d-42fa-bf4c-3813c2111c30 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:57.621701 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20ec1c6-8720-43d1-919d-159bffb89147 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e782c8acaaa42c4bd78ddc7c8cc654b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:57.622298 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ec1c6-8720-43d1-919d-159bffb89147 service cinder] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:57.622577 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ec1c6-8720-43d1-919d-159bffb89147 service cinder] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:57.622848 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ec1c6-8720-43d1-919d-159bffb89147 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:57.653942 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20ec1c6-8720-43d1-919d-159bffb89147 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:57.653942 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20ec1c6-8720-43d1-919d-159bffb89147 service cinder] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:57.653942 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1477/2923] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 23:00:57 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:01:07.543291 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=71,keystone:INSERT=3 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:01:07.637863 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:DELETE=1,keystone:SELECT=55,keystone:INSERT=2 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:01:09.098642 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da907903-7f39-4ba5-ae13-fc2ec1c5a625 None None] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:01:09.171060 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da907903-7f39-4ba5-ae13-fc2ec1c5a625 None None] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:01:09.175793 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505b350-eccb-45f9-97da-8efc4d033990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:01:09.176420 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505b350-eccb-45f9-97da-8efc4d033990 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:01:09.176757 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505b350-eccb-45f9-97da-8efc4d033990 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:01:09.177202 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505b350-eccb-45f9-97da-8efc4d033990 admin admin] PATH_INFO: `/v3/users/bcd7a111086c4564bd86c09b7c859358` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:01:09.178565 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d505b350-eccb-45f9-97da-8efc4d033990 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcd7a111086c4564bd86c09b7c859358)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:01:09.180876 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d505b350-eccb-45f9-97da-8efc4d033990 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:01:09.369071 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-d505b350-eccb-45f9-97da-8efc4d033990 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcd7a111086c4564bd86c09b7c859358'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:01:09.384766 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-d505b350-eccb-45f9-97da-8efc4d033990 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcd7a111086c4564bd86c09b7c859358'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:01:09.394423 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-d505b350-eccb-45f9-97da-8efc4d033990 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcd7a111086c4564bd86c09b7c859358'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:01:09.444920 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-d505b350-eccb-45f9-97da-8efc4d033990 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcd7a111086c4564bd86c09b7c859358'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:01:09.445567 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-d505b350-eccb-45f9-97da-8efc4d033990 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcd7a111086c4564bd86c09b7c859358'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:01:09.453270 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1447/2924] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 23:01:09 2022] DELETE /identity/v3/users/bcd7a111086c4564bd86c09b7c859358 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:01:09.458896 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1f5d0-e843-48ac-87b6-9a4afdd7b6ad None None] REQUEST_METHOD: `POST` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:01:09.459438 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1f5d0-e843-48ac-87b6-9a4afdd7b6ad None None] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:01:09.459639 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1f5d0-e843-48ac-87b6-9a4afdd7b6ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:01:09.483042 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.auth.core [None req-85e1f5d0-e843-48ac-87b6-9a4afdd7b6ad None None] MFA Rules not processed for user `9cc82f35b5c947928c9caa5d0b7e11fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:01:09.484266 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-85e1f5d0-e843-48ac-87b6-9a4afdd7b6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:01:09.560378 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1478/2925] 10.222.0.52 () {58 vars in 958 bytes} [Mon Aug 29 23:01:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:01:09.575048 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505b350-eccb-45f9-97da-8efc4d033990 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:01:09.576473 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-d505b350-eccb-45f9-97da-8efc4d033990 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:01:09.653383 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505b350-eccb-45f9-97da-8efc4d033990 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:01:09.657545 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f09be60-00c5-4960-8e3b-2dff9b256c79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:01:09.658147 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f09be60-00c5-4960-8e3b-2dff9b256c79 service neutron] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:01:09.658424 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f09be60-00c5-4960-8e3b-2dff9b256c79 service neutron] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:01:09.658691 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f09be60-00c5-4960-8e3b-2dff9b256c79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:01:09.668140 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f09be60-00c5-4960-8e3b-2dff9b256c79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:01:09.669659 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f09be60-00c5-4960-8e3b-2dff9b256c79 service neutron] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:01:09.682403 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1448/2926] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 23:01:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:01:10.087557 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e1f5d0-e843-48ac-87b6-9a4afdd7b6ad None None] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:01:10.089223 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-85e1f5d0-e843-48ac-87b6-9a4afdd7b6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:01:10.101318 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e1f5d0-e843-48ac-87b6-9a4afdd7b6ad None None] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:01:10.106383 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edc1e63-3f4e-416c-b313-23839081a3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:01:10.107026 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edc1e63-3f4e-416c-b313-23839081a3ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:01:10.108052 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edc1e63-3f4e-416c-b313-23839081a3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:01:10.108348 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edc1e63-3f4e-416c-b313-23839081a3ab admin admin] PATH_INFO: `/v3/projects/260ffd54bf93432691ee9bdc1bf96139` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:01:10.109716 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edc1e63-3f4e-416c-b313-23839081a3ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=260ffd54bf93432691ee9bdc1bf96139)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:01:10.111963 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edc1e63-3f4e-416c-b313-23839081a3ab admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:01:10.263218 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-4edc1e63-3f4e-416c-b313-23839081a3ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '260ffd54bf93432691ee9bdc1bf96139', 'request_id': 'req-4edc1e63-3f4e-416c-b313-23839081a3ab'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:01:10.269849 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-4edc1e63-3f4e-416c-b313-23839081a3ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '260ffd54bf93432691ee9bdc1bf96139', 'request_id': 'req-4edc1e63-3f4e-416c-b313-23839081a3ab'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:01:10.276411 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-4edc1e63-3f4e-416c-b313-23839081a3ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '260ffd54bf93432691ee9bdc1bf96139', 'request_id': 'req-4edc1e63-3f4e-416c-b313-23839081a3ab'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:01:10.374007 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-4edc1e63-3f4e-416c-b313-23839081a3ab admin admin] The token cache is being invalidate because project 260ffd54bf93432691ee9bdc1bf96139 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:01:10.374295 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-4edc1e63-3f4e-416c-b313-23839081a3ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:01:10.374915 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-4edc1e63-3f4e-416c-b313-23839081a3ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:01:10.377139 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1479/2927] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 23:01:10 2022] DELETE /identity/v3/projects/260ffd54bf93432691ee9bdc1bf96139 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:01:19.676821 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:01:20.370801 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:01:32.457589 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f09be60-00c5-4960-8e3b-2dff9b256c79 service neutron] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:01:32.459366 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-5f09be60-00c5-4960-8e3b-2dff9b256c79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:01:32.530672 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f09be60-00c5-4960-8e3b-2dff9b256c79 service neutron] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:01:32.592279 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30db578-751d-4ed5-af8f-c1b576bfbde9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8fd0f6e5b144d2ca78b8d96722502a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:01:32.592928 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30db578-751d-4ed5-af8f-c1b576bfbde9 service placement] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:01:32.593217 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30db578-751d-4ed5-af8f-c1b576bfbde9 service placement] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:01:32.593489 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30db578-751d-4ed5-af8f-c1b576bfbde9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:01:32.595246 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-b30db578-751d-4ed5-af8f-c1b576bfbde9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:01:32.607338 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30db578-751d-4ed5-af8f-c1b576bfbde9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:01:32.609001 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30db578-751d-4ed5-af8f-c1b576bfbde9 service placement] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:01:32.618868 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1449/2928] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 23:01:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:01:42.614670 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=58 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:02:22.424361 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edc1e63-3f4e-416c-b313-23839081a3ab admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:02:22.425805 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-4edc1e63-3f4e-416c-b313-23839081a3ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:02:22.486701 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edc1e63-3f4e-416c-b313-23839081a3ab admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:02:22.490410 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af279a-ccf1-47db-9f53-b313724cca38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fe5e8c4cb324ffb9d5b80f18d123727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:02:22.491018 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af279a-ccf1-47db-9f53-b313724cca38 service neutron] REQUEST_METHOD: `GET` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:02:22.491300 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af279a-ccf1-47db-9f53-b313724cca38 service neutron] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:02:22.491577 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af279a-ccf1-47db-9f53-b313724cca38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:02:22.493408 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-82af279a-ccf1-47db-9f53-b313724cca38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:02:22.569530 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82af279a-ccf1-47db-9f53-b313724cca38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:02:22.571883 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82af279a-ccf1-47db-9f53-b313724cca38 service neutron] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:02:22.585792 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1480/2929] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 23:02:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:02:29.991310 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30db578-751d-4ed5-af8f-c1b576bfbde9 service placement] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:02:29.992909 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-b30db578-751d-4ed5-af8f-c1b576bfbde9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:02:30.008479 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30db578-751d-4ed5-af8f-c1b576bfbde9 service placement] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:02:30.011809 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d3713c-034d-4947-bc41-218ec7d33170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:02:30.012429 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d3713c-034d-4947-bc41-218ec7d33170 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:02:30.013422 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d3713c-034d-4947-bc41-218ec7d33170 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:02:30.013422 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d3713c-034d-4947-bc41-218ec7d33170 admin admin] PATH_INFO: `/v3/users/df23a24f4d5948deb641de1cefad1f69` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:02:30.014627 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d3713c-034d-4947-bc41-218ec7d33170 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df23a24f4d5948deb641de1cefad1f69)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:02:30.016529 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d3713c-034d-4947-bc41-218ec7d33170 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:02:30.251222 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-c0d3713c-034d-4947-bc41-218ec7d33170 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df23a24f4d5948deb641de1cefad1f69'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:02:30.264027 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-c0d3713c-034d-4947-bc41-218ec7d33170 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df23a24f4d5948deb641de1cefad1f69'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:02:30.270875 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-c0d3713c-034d-4947-bc41-218ec7d33170 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df23a24f4d5948deb641de1cefad1f69'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:02:30.413701 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-c0d3713c-034d-4947-bc41-218ec7d33170 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df23a24f4d5948deb641de1cefad1f69'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:02:30.414411 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-c0d3713c-034d-4947-bc41-218ec7d33170 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df23a24f4d5948deb641de1cefad1f69'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:02:30.421378 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1450/2930] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 23:02:29 2022] DELETE /identity/v3/users/df23a24f4d5948deb641de1cefad1f69 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:02:30.666982 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af279a-ccf1-47db-9f53-b313724cca38 service neutron] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:02:30.677897 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af279a-ccf1-47db-9f53-b313724cca38 service neutron] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:02:30.739208 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee820f4d-4f8e-4596-97d1-fc687016a039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:02:30.739835 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee820f4d-4f8e-4596-97d1-fc687016a039 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:02:30.740115 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee820f4d-4f8e-4596-97d1-fc687016a039 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:02:30.740390 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee820f4d-4f8e-4596-97d1-fc687016a039 admin admin] PATH_INFO: `/v3/projects/98941ac6692045859aa7639192603764` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:02:30.741536 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee820f4d-4f8e-4596-97d1-fc687016a039 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98941ac6692045859aa7639192603764)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:02:30.743446 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee820f4d-4f8e-4596-97d1-fc687016a039 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:02:30.905852 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ee820f4d-4f8e-4596-97d1-fc687016a039 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98941ac6692045859aa7639192603764', 'request_id': 'req-ee820f4d-4f8e-4596-97d1-fc687016a039'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:02:30.907682 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:02:30.911387 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ee820f4d-4f8e-4596-97d1-fc687016a039 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98941ac6692045859aa7639192603764', 'request_id': 'req-ee820f4d-4f8e-4596-97d1-fc687016a039'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:02:30.917450 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ee820f4d-4f8e-4596-97d1-fc687016a039 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98941ac6692045859aa7639192603764', 'request_id': 'req-ee820f4d-4f8e-4596-97d1-fc687016a039'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:02:30.995715 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ee820f4d-4f8e-4596-97d1-fc687016a039 admin admin] The token cache is being invalidate because project 98941ac6692045859aa7639192603764 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:02:30.996003 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ee820f4d-4f8e-4596-97d1-fc687016a039 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:02:30.996628 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-ee820f4d-4f8e-4596-97d1-fc687016a039 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:02:30.998173 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1481/2931] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 23:02:30 2022] DELETE /identity/v3/projects/98941ac6692045859aa7639192603764 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:02:31.002804 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d3713c-034d-4947-bc41-218ec7d33170 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:02:31.004260 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-c0d3713c-034d-4947-bc41-218ec7d33170 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:02:31.017694 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d3713c-034d-4947-bc41-218ec7d33170 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:02:31.076103 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b260f8-48aa-4b14-8499-f21fc084f2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:02:31.076747 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b260f8-48aa-4b14-8499-f21fc084f2a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:02:31.077038 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b260f8-48aa-4b14-8499-f21fc084f2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:02:31.077317 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b260f8-48aa-4b14-8499-f21fc084f2a2 admin admin] PATH_INFO: `/v3/users/b2a7c52332be42f19d9a2130b3f0caec` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:02:31.078465 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b260f8-48aa-4b14-8499-f21fc084f2a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2a7c52332be42f19d9a2130b3f0caec)` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:02:31.080368 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b260f8-48aa-4b14-8499-f21fc084f2a2 admin admin] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:02:31.307343 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-e3b260f8-48aa-4b14-8499-f21fc084f2a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2a7c52332be42f19d9a2130b3f0caec'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:02:31.319546 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-e3b260f8-48aa-4b14-8499-f21fc084f2a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2a7c52332be42f19d9a2130b3f0caec'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:02:31.327328 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-e3b260f8-48aa-4b14-8499-f21fc084f2a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2a7c52332be42f19d9a2130b3f0caec'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:02:31.329059 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:02:31.424915 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-e3b260f8-48aa-4b14-8499-f21fc084f2a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2a7c52332be42f19d9a2130b3f0caec'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:02:31.425551 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.notifications [None req-e3b260f8-48aa-4b14-8499-f21fc084f2a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2a7c52332be42f19d9a2130b3f0caec'} {{(pid=93272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:02:31.436608 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1451/2932] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Aug 29 23:02:31 2022] DELETE /identity/v3/users/b2a7c52332be42f19d9a2130b3f0caec => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:02:31.677177 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee820f4d-4f8e-4596-97d1-fc687016a039 admin admin] Authenticating user token {{(pid=93271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:02:31.678665 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.fernet_utils [None req-ee820f4d-4f8e-4596-97d1-fc687016a039 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:02:31.692698 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee820f4d-4f8e-4596-97d1-fc687016a039 admin admin] Validating token access rules against request {{(pid=93271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:02:31.755384 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1238c93f-f19c-4a5e-bc23-560f4fd17e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cc82f35b5c947928c9caa5d0b7e11fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb4839200e8e45dfa0acd9e9d8f433bc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:02:31.756015 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238c93f-f19c-4a5e-bc23-560f4fd17e94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:02:31.756295 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238c93f-f19c-4a5e-bc23-560f4fd17e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:02:31.756567 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238c93f-f19c-4a5e-bc23-560f4fd17e94 admin admin] PATH_INFO: `/v3/projects/e631bf7db91048f88772928b60dec182` {{(pid=93271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:02:31.757712 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1238c93f-f19c-4a5e-bc23-560f4fd17e94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e631bf7db91048f88772928b60dec182)` {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:02:31.759604 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1238c93f-f19c-4a5e-bc23-560f4fd17e94 admin admin] RBAC: Authorization granted {{(pid=93271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:02:31.865115 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-1238c93f-f19c-4a5e-bc23-560f4fd17e94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e631bf7db91048f88772928b60dec182', 'request_id': 'req-1238c93f-f19c-4a5e-bc23-560f4fd17e94'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:02:31.870615 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-1238c93f-f19c-4a5e-bc23-560f4fd17e94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e631bf7db91048f88772928b60dec182', 'request_id': 'req-1238c93f-f19c-4a5e-bc23-560f4fd17e94'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:02:31.877300 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-1238c93f-f19c-4a5e-bc23-560f4fd17e94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e631bf7db91048f88772928b60dec182', 'request_id': 'req-1238c93f-f19c-4a5e-bc23-560f4fd17e94'} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:02:31.995613 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-1238c93f-f19c-4a5e-bc23-560f4fd17e94 admin admin] The token cache is being invalidate because project e631bf7db91048f88772928b60dec182 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:02:31.995827 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-1238c93f-f19c-4a5e-bc23-560f4fd17e94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:02:31.996402 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG keystone.notifications [None req-1238c93f-f19c-4a5e-bc23-560f4fd17e94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:02:31.998590 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: [pid: 93271|app: 0|req: 1482/2933] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Aug 29 23:02:31 2022] DELETE /identity/v3/projects/e631bf7db91048f88772928b60dec182 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:02:41.527431 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:02:41.992236 dv-cinder-854269-4-iscsi devstack@keystone.service[93271]: DEBUG dbcounter [-] [93271] Writing DB stats keystone:SELECT=57,keystone:DELETE=9,keystone:INSERT=2 {{(pid=93271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:06:34.387366 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b260f8-48aa-4b14-8499-f21fc084f2a2 admin admin] Authenticating user token {{(pid=93272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:06:34.388866 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-e3b260f8-48aa-4b14-8499-f21fc084f2a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:06:34.529253 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b260f8-48aa-4b14-8499-f21fc084f2a2 admin admin] Validating token access rules against request {{(pid=93272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:06:34.590469 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621932c9-0d7a-4e75-bf68-1eb06cac9a4e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8fd0f6e5b144d2ca78b8d96722502a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee84c3340f2641ad963655a7a212ef96', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:06:34.591078 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621932c9-0d7a-4e75-bf68-1eb06cac9a4e service placement] REQUEST_METHOD: `GET` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:06:34.591360 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621932c9-0d7a-4e75-bf68-1eb06cac9a4e service placement] SCRIPT_NAME: `/identity` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:06:34.591639 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621932c9-0d7a-4e75-bf68-1eb06cac9a4e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:06:34.593451 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.fernet_utils [None req-621932c9-0d7a-4e75-bf68-1eb06cac9a4e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:06:34.605989 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621932c9-0d7a-4e75-bf68-1eb06cac9a4e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:06:34.607572 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621932c9-0d7a-4e75-bf68-1eb06cac9a4e service placement] RBAC: Authorization granted {{(pid=93272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:06:34.618493 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: [pid: 93272|app: 0|req: 1452/2934] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Aug 29 23:06:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:06:44.613252 dv-cinder-854269-4-iscsi devstack@keystone.service[93272]: DEBUG dbcounter [-] [93272] Writing DB stats keystone:SELECT=89 {{(pid=93272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}